I’ve uploaded the latest release of my emacs dotfiles. Download them here:

Adam’s Emacs Dotfiles

From the changelog:

2006-09-03
– ruby 1.74.2.14 from Ruby 1.8.4
– ruby electric Mar 2005 from Dee Zsombor
– many ruby fixes and enhancements (Rakefile, yaml, indent=2, etc.)
– cc-mode 5.31.3
– csharp-mode 0.5.0 from Dylan Moonfire (thanks Brigham)

To build on my previous post concerning Windows apps, I’d like to suggest some Firefox extensions for your enjoyment. I think that my small readership will appreciate this list, especially those who complained that my previous post simply did not apply to their Mac-centric lives.

My current Firefox install contains 19 extensions. Instead of listing the most popular (adblock, web developer, etc.) I will instead give you the more obscure (yet handy) extensions that I’ve encountered:

• Colorful Tabs – Colors the tabs in Firefox. Looks great. The author’s homepage features a bizarre circa-1997 FRAME layout, which makes navigation exciting. If you can find the installer you’re in for a treat.
• Download Statusbar – Rid yourself of that annoying download window. Instead, download progress will quietly show up in the status bar. I especially like the gradient progress bars.
• Duplicate Tab – Right click on a tab and select “Duplicate Tab”. Why isn’t this built into Firefox?
• FindBar Switcher – Hit CTRL-F to make the find bar appear and disappear. Another one for the “soon to be included in Firefox” list.
• Fission – Turns your bland white address bar background into a sleek blue progress bar. No need to switch now!
• Google Pagerank Status – Displays the Google PageRank in the status bar. You SEO fiends can finally uninstall that clunk Google Toolbar extension.
• Live HTTP Headers – Displays the HTTP headers sent and received by Firefox while you browse. I’ve devoted multiple weeks of my life to writing various debug proxies. Happily, all my hard work is obsoleted by this simple extension.
• NextPlease! – Many sites slice a single article across multiple pages, either to artificially inflate ad views or reduce server load. NextPlease! let’s you navigate to the “next page” using a keyboard shortcut. Works with nytimes, gamespot, google, tom’s hardware, ebay, google… The list goes on and on. Great!
• Super DragAndGo – Click and drag to “throw” a link toward the top of the screen and open it in a new tab. I use this dozens of times each day. Indispensable.
• TableTools – Windows really spoiled us by making most tables easily sortable. The fact that many HTML tables still can’t be sorted is a sad testament to the complexities of CSS, HTML and SQL. The TableTools extension offers lots of strange and wonderful features which I don’t use, but I adore its ability to sort tables.

I have a soft spot for well written Windows apps, especially when they’re small and free. Let’s raise a cheer for the legions of hackers struggling to make the world’s most popular OS usable. Do yourself a favor and install the following:

• AutoRuns – list (and potentially turn off) applications that run at startup. Take that Quicktime and RealPlayer!
• AutoStitch – stitch multiple photos together into a panorma. You might want this if you visit places like Mt. Rainier, Sydney, etc.
• Console – an excellent command line “console” for windows in which you can run cmd.com (or tcsh, or bash, or whatever). Supports tabs, transparency, wild color schemes, and any monospaced font your heart desires.
• DVD Decrypter – turn DVDs into ISOs and vice versa. I believe development ceased because this is considered illegal according to the DMCA. Luckily, the current release is almost perfect.
• Easy Thumbnails – batch image processing. You can easily resize a bunch of photos for inclusion on a web page. For some reason I have to do this quite often.
• Ffdshow – play all video codecs on Windows. Also, get Media Player Classic and expunge Windows Media Player forever.
• Mp3tag – clean up the tags in your mp3 files, then reimport into iTunes. You’ll thank me later.
• Notepad++ and Notepad2 – two great options for quick edits. They’re tiny, start instantly, and highlight lots of different file formats.
• Picasa – for organizing and fixing up photos. Better yet, try the recently released Picasa Beta, which finally includes a Save Changes button for when you’re done retouching. Incidentally, if you’re interested in graphics hacks you should google Michael Herf, who used to be Picasa’s CTO before they were bought by google. His stereopsis page is fascinating.
• Taskbar Shuffle – drag and drop the buttons in your taskbar. Where have you been all these years?

I’ll be discussing Bash on Windows (and MSYS) in a future post.

The Fool’s Errand was a Macintosh game released in 1988 by Cliff Johnson, who went on to create several other moderately successful puzzle games. None of his subsequent efforts could top the brilliance of The Fool’s Errand.

The Fool’s Errand is the story of a wandering fool tasked with saving the land. The game consists of a set of linked puzzles, tightly coiled together with a Tarot theme. Along the way you encounter each character from the Taro deck, and you must help The Fool unlock their secrets before you can proceed. Portions of the map can’t be reached until certain puzzles have been completed, and the final puzzle cannot be attempted until all others are finished. Strangely, none of the puzzles include instructions. Figuring out the rules is part of the fun.

As a thirteen year old I found the game exceptionally challenging. Now that I’m partially grown, I thought I’d give it another go to see how my skills have improved. It was still rather strenuous. Luckily I was able to vaguely recall some of the puzzles – otherwise I might have been stuck on the The Humbug indefinitely.

Here are a few screen shots:

If you want to play The Fool’s Errand on Windows, try the following:

1. Download Mini vMac.
2. Rip the ROM from an existing Mac Classic, or try googling file:vmac.rom. Drop the ROM file into the same directory as vMac.
3. Laboriously create a bootable Macintosh disk image using HFVExplorer.
4. With a last Herculean effort, copy The Fool’s Errand files onto your new boot disk. Without losing the resource forks.

Or, you can skip all that hard work and just download a playable copy of the above. Drag fool.dsk onto vMac to start. Have fun!

An Aside : Cliff Johnson

I’m grateful to Cliff for creating The Fool’s Errand and helping me fill those endless days while I awaited puberty. Still, I find his home page somewhat disquieting and I can’t help but feel that Cliff is teetering on the edge of madness. Cliff has been working hard on The Fool and his Money, a sequel to The Fool’s Errand. I pre-ordered the game a year or two ago, and despite numerous delays I’m eagerly anticipating its imminent release.

The Problem

Recently, I was called in to review the design for a major piece of functionality in a soon-to-be-shipping product. What began as an inspection eventually grew into a complete rewrite. The most glaringly awful code dealt with storing key/value properties on disk. The code was so overgrown that we couldn’t even begin to consider replacing it for this release. The property code was on the periphery of my original assignment, but it’s sheer size compelled me to take a look.

The property code was amazing! It allowed for categories, so that different sets of properties were guaranteed not to collide. Property bags could be intelligently merged and combined. Each property was typed, with massive amounts of type checking and assertions galore. The properties themselves were persisted on disk using multiple file formats simultaneously. The writable version was stored using sqlite while the readonly version used a custom format complete with a lex/yacc parser.

The implementation was 6,000 lines of code scattered across a dozen files. Nobody wanted to change the code because it was so inscrutable. Most of the features had never been tested. The property code was a ticking time bomb, but it was too late to make any changes for the current release.

It’s Just a Hashtable

This mess could easily be replaced with a string to string hashtable. Hashtables are simple to implement, use, and test, even in a washed up language like C. Hashtables can easily be persisted to text, XML, URL query strings, RPC calls, and databases. Just to be clear:

• Instead of schema checking, use plain old strings. Convert the values to the appropriate type as necessary.
• To merge hashtables, add a putAll() method that adds one hashtable to another.
• To avoid collisions, prepend a category string to each key. You can put the prefix functionality into a subclass.
• Add read/write methods to persist your hashtable.

This situation would be comical if I hadn’t seen it so often at so many different companies. People inevitably seem to think that hashtables aren’t efficient or type safe enough for their specific needs. I’ve implemented nearly identical hashtable wrappers in C, C++, Java, Ruby, Perl, and Python. I’ve run it on Linux and Windows, on ARM, MIPS, and x86. I feel a warm sense of security each time I write those familiar lines because IT’S JUST A HASHTABLE.

Be Lazy

The lazy programmer uses hashtables wherever possible. If you think you can do better than a hashtable, think twice. If you think you need type safety for property files, please reconsider. If you think hashtables take too much memory or are too slow, you’re probably wrong.

Be lazy and use hashtables. You can always churn out those 6,000 lines later if the need arises.

Today I am releasing my emacs dotfiles for public consumption. Download them here : Adam’s Emacs Dotfiles

History

At Strangeberry we faced a dilemma. Arthur, Jonathan and I had deep experience with emacs, but our dotfiles were wildly divergent. We decided to combine our skills and create a rich set of dotfiles that was also easy to customize. Over time, these dotfiles grew in popularity and are now used by many people across several different companies. Thanks to all the people who contributed to this effort – Arthur van Hoff, Jonathan Payne, Adam Feder, Greg Spencer, Brigham Stevens, and Jeff Jolma.

Features

Here is a brief feature list:

• cross platform – works on Windows and Linux, windowed or in a terminal
• lots of modes – including some simple ones I created (valgrind, svn, mbox, NSIS, …)
• customizable – these dotfiles are probably more customizable then others you’ve seen, because they’ve been used by so many people over the years
• speedy – I use autoloads everywhere so emacs starts up quickly
• make.el – helpers for compiling (try M-s)
• abtags.el – quickly jump around large numbers of files. Build a TAGS file with ctags, then use abtags-find-file (bound to F7 in my .emacs).
• TAGS helpers – M-. to lookup a tag, M-, to pop back

If there is enough public interest I will release subsequent versions as my own dotfiles evolve. Enjoy!

C is an obsolete language and there’s really no good reason to use it. For all but the most esoteric projects, C++ should be used instead of C. I may elaborate on this theme in a future post.

That being said, sometimes you’re stuck with C code. Maybe you’re working on legacy code, or you don’t have control over the engineering requirements. Maybe management mistakenly thought C was “faster” or “more portable”. Maybe there’s no C++ compiler for your platform.

Recently, I was asked to reduce the memory footprint for a C application which had somehow ballooned to gargantuan proportions. The application was targeted at an embedded device with 64MB of RAM and no virtual memory. A bit of sleuthing revealed that a geocoding API was allocating a 20MB data structure. That’s a pretty hefty chunk of change for our little device. The API allocated space for up to 50 matches (each of which was 400kb), instead of using a dynamic array to return the results. The code was written by a junior engineer.

When using C, you tend to spend inordinate amounts of time managing arrays, strings, and hashtables. Think of the thousands of hours wasted on such minutae. Isn’t our collective time more valuable? I feel pity for the poor engineers (like me) forced to labor under such cruel conditions.

Here is a recipe for quick and dirty dynamic arrays in C. I have used this pattern many times with excellent results.

Introduction

This example is targeted at a specific use case. It will serve you well if you have to accumulate an unknown amount of data. You can roll this into an existing struct or grow it into a generic void* array class. I tend to use it inline in other data structures because it’s so simple. You can also use this pattern in Java if you want to avoid the overhead of the java.util.* classes.

Struct

First, create a struct or add the members to an existing struct. In this case I’m creating a new struct and my array will store char*’s. Note that pos is the number of elements in the array, and capacity is the number of elements that the array is capable of storing. Feel free to use other variable names if they make more sense to you.

Gub is simply a word I use a lot in test code, similar to foo or bar.

typedef struct {
    int pos;
    int capacity;
    char **array;
} Gub;

Init the struct

calloc is a handy allocation function that fills the memory with zeroes before returning it. This is an easy way to avoid uninitialized memory errors. In this case, I’m also counting on that behavior because I want pos to be 0 and I want array to be NULL before proceeding.

Gub *g = (Gub*)calloc(sizeof(Gub), 1);

Create an empty array

When creating the empty array, it’s necessary to select an initial capacity. In this case the array can hold four elements before it needs to be resized. Note that we init capacity inline here. Personally I think this makes the code more readable.

This could be written just as easily with malloc (or calloc), but I prefer to use realloc to matche the code used when appending elements (see below). Realloc is a magical allocation function that resizes an existing block of allocated memory to a new size. It’s tremendously useful for resizable data structures. If you pass NULL to realloc (as we’re doing here with the g->array), realloc will simply allocate new memory for you.

g->array = (char**)realloc(g->array, (g->capacity = 4) *
                                     sizeof(char*));

Append some data

This is a silly loop that appends argv to the array. Before adding an element to the array, we check to see if there’s room for more elements. If pos == capacity, the array is full and must be resized. Realloc resizes the block of memory and then we’re good to go. Again, note that capacity is modified inline during the realloc. I think this makes the code more readable. In this case I’m simply doubling the size of the array whenever we run out of space.

Also, note that g->pos is incremented inline when we actually add the element.

for (int i = 0; i < argc; ++i) {
    if (g->pos == g->capacity) {
        g->array = (char**)realloc(g->array, (g->capacity *= 2) *
                                             sizeof(char*));
    }
    g->array[g->pos++] = argv[i];
}

Free

When we’re done with the array, we free it and then the containing struct. Don’t forget to free the elements themselves if necessary.

free(g->array);
free(g);

I am an exceptionally lazy developer. If you look at my code, you’ll find that I mercilessly cut corners, fail to implement obvious features, and my big O sometimes careens wildly out of control. Why am I so lazy?

When I began my career I was the opposite of lazy. I tried to future-proof my code by designing complicated systems that anticipated our undefined needs. I carefully scrutinized every loop and allocation for stray operations that would impact performance. Like a freshman vying for extra credit, I piled on features like there was no tomorrow. Gradually, I came to understand that my efforts were misguided:

• future-proofing rarely works, because the product inevitably evolves in unforseen ways. You’re likely to end up with a bridge to nowhere. Also, the wacky futuristic code that you write will be untestable and buggy. Future-proofing is an example of over engineering, where well-meaning engineers mess things up by trying too hard.
• premature optimization is another no-no. Optimized code is more difficult to write, debug, and test, and always ends up buggier than unoptimized code. Just turning on optimization in the compiler can sometimes lead to horrible bugs. Instead of optimizing every line, find and fix the bottlenecks. It’s more satisfying that way anyhow.
• extra features probably won’t earn you any bonus points. Your features might not get used, and you’re definitely creating more bugs for your poor QA team to squash. This is another example of over engineering.

Writing software is a constant battle against complexity. Complexity kills. Complexity kills testing, deadlines, products, careers, and companies. Each additional line of code increases complexity. In the pursuit of excellence, senior developers must constantly strive to eliminate features that aren’t absolutely necessary.

Cultivating laziness is the easiest way to combat complexity. The lazy developer doesn’t need to show off by tossing more features onto the pile. The lazy developer optimizes intelligently instead of trying to perfect each line. The lazy developer hesitates before deploying complex and indecipherable algorithms.

By now most of you are probably nodding your heads (or nodding off). But clearly there are people who don’t get it, because I’m constantly running into complicated, over engineered software written by developers who are trying too hard.

Future posts will illustrate with some anecdotes. Names will be omitted to protect the guilty.

A few weeks ago I created a data feed in the same vein as Paul Kedrosky’s (now defunct) BubbleWire. The Coming Soon feed lists “interesting” domain names that will soon expire. For my purposes, “interesting” means that the domain name is a common english word. Here’s the FeedBurner link:

http://feeds.feedburner.com/gurge/coming_soon

For the technically proficient among you, here’s how Coming Soon is generated:

1. Each night a Ruby script fires up on my linux box.
2. The script fetches the list of expiring domains from pool.com. You can find the link on their home page.
3. Each domain is given a quality ranking.
4. For the time being I’m only interested in domains with QUALITY=1, which means that the domain is in the 3esl list.
5. The interesting domains are placed in a db.
6. The RSS feed is generated and uploaded to gurge.com, where it gets picked up by feedburner.

I have several other quality rankings:

• is the domain an uncommon word?
• is the domain two common words put together?
• is the domain two uncommon words put togehter?
• etc.

I found that the other lists are way too long to be useful for casual domain watchers like me.

Putting together a PC from scratch is fun and not too difficult. Line things up, put in a few screws and flip the switch. If you hear the happy POST beep, you’re good to go.

Installing the OS is another story. I’ve fought (and won) several epic battles with various linux distros, trying to convince them to accept common PC hardware. Hopefully we’re near the end of the “compile your own kernel and drivers” era. On the other hand, Windows always installs perfectly. Locating and installing drivers is generall pretty easy since hardware companies target Windows as their main platform.

This time, things were different. I’m embarassed to say that I ignored the warnings on the newegg message board. I figured they were out of date, or uninformed. In retrospect I should have paid far more attention to things like this in the newegg reviews:

“ASRock 939SLI32-eSATA ATX AMD Motherboard

Pros: …

Cons: Sata/Sata Raid has to have a floppy for f6 (not included), but the drivers CD is bootable and has will make the floppy with easy to follow instructions.”

Let me translate this into something more meaningful for the casual reader.

When you try to install Windows XP onto your brand new SATA RAID array, Windows won’t recognize the drive. Googling indicates that you have to press F6 during an obscure 3-second interval of the initial XP install process. A prompt flashes on screen for a few milliseconds to inform you of this fact.

If you manage to hit F6 quickly enough, Windows will then ask you to insert a floppy with the correct SATA drivers. Not a CD or USB key, a floppy. A floppy?!?! Who the hell has floppy drives anymore, especially in a home built system?

So, you run out to the nearest store (Computer Stop on Aurora) to buy a floppy drive for ~$15. When you get home, you have to install the floppy drive into your old PC in order to copy the SATA drivers to a disk.. Let’s hope you haven’t yet taken apart the old PC. Unfortunately, it’s impossible to find the SATA drivers online. You have to use the magic install CD that came with the motherboard. Let’s also hope you haven’t taken the CDROM out of your old PC!

Don’t forget to detach the floppy from the old PC and drop it into the new PC.
Assuming you’ve done everything correctly, and you have good reflexes, you will now be able to press F6 and tell Windows about the drivers so it can talk to your RAID array. Luckily I played a lot of Egyptian Ratscrew as a child so that part wasn’t too much of a problem. Windows will now install perfectly.

Time spent assembling the new PC : 20 minutes
Time spent getting XP to recognized my RAID array : 6 hours