Project Reclaim and the email dilemma

One of the main 2011 goals for Project Reclaim is to get my email out of Gmail. Heavy reliance on Gmail raises a number of red flags. For one thing, email is central to my business and personal life online, and provides the best archive of my online past (get the important stuff first). For another, Gmail is ad-supported, in a way that has rankled since Gmail went public: it “reads” your email and serves ads based on what it finds. No one really talks about it anymore, but it still kind of bugs me – so I want to move to a non-free system (paying is better than getting something for free).

It’s taken me a while to make the move, though, for two main reasons.

  1. Email is tricky. Good, free mail server software is easy to find. But it’s not necessarily easy to set up and maintain. If the outgoing server isn’t configured correctly, your messages will get marked as spam. If you haven’t got constantly monitored spam filters on your incoming mail, you’ll be inundated with garbage. And the issues of backups and reliability, while certainly important in the case of (say) self-hosted websites, are many times more important with email: if the server goes down, emails may get altogether lost in the ether.

    I’ve set up and configured email servers before, and it hasn’t been very fun. When deciding how to solve the Gmail conundrum, I needed to take this fact into consideration. I started to do a bit of research on paid email hosting, and found good reviews of Rackspace’s hosted email service. The service is pretty affordable, and I knew from years of Slicehost use (now owned by Rackspace) that customer service and support would be good.

  2. I needed a good address. I own a lot of domain names, but most of them are lame, and none lent themselves very neatly to an email address. For instance, when your domain name is boonebgorges.com, what’s the email account name? ‘boone’? The cool factor there is pretty low. And I am a cool guy, so that’s important.

    Some of the obvious domains are taken. boone.com is wasted on dry-erase boards. gorges.com could never be wrested from the clutches of “one of the oldest family owned Volvo franchises in the United States”. But there was hope – or should I say habĂ­a esperanza – that I might get the fairly unused gorg.es. In fact, my brother and I had been working on that project for a couple of years, but it was only a few months ago that the owner finally relented, and the domain name was transferred to the Gorges boys.

So, about two months ago, I made the switch. For now, I just set it up as another account in Thunderbird (more on my Thunderbird setup). I created a generic “Archive” directory on my gorg.es account (to mimic Gmail’s All Mail) and pointed my ‘Y’ shortcut to that directory. I’m using K-9 Mail on my Android phone, which I set up to save the entire Archive directory, so I’d have good local email search on my phone. Little by little, I’m moving over my email correspondence to the new, awesome address. Bye bye, Gmail!

Making the Thunderbird interface more Gmail-y

As part of Project Reclaim, I’m gearing up to move my email off of Gmail and onto my own server. Email is, and long has been, central to my life online – it’s my main point of contact for so many personal and professional connections, and my email archives are the closest thing I’ve got to a record of my online activity. So I’m keen to make the move as smooth as I can.

For that reason, I’m handling the transition in stages. The first stage involves transitioning email client software away from the Gmail website.

Choosing a client

I’ve chosen Thunderbird as my alternative, for a couple reasons.

  1. It’s open-source.
  2. It’s highly extensible and customizable.
  3. It works across platforms. That’s important, because I’ll be migrating away from OSX.

Choosing Thunderbird is not without its sacrifices. For one thing, moving to an OS-native application, rather than an app that runs in a browser window like Gmail does, means that I’ll no longer be able to count on having a consistent UI and feature set between different devices. In some cases, this is not a huge loss. The UI for Gmail on my Android phone is really quite different from the normal web version, and it’s never bothered me very much. My biggest worry, though, is that I’ll have multiple workstations – a primary work machine and a netbook, for example – with different email setups. I’m hoping to mitigate the problem by coming up with some idiot-proof backup and syncing methods for the fairly small number of files that comprise Thunderbird’s configuration. (This’ll be necessary for other software transitions as well, like my gradual move to Vim. It’d be quite easy with Dropbox and some strategically-placed symbolic links, but I’m trying to break the Dropbox habit too :) )

It’s a considerable comfort that, now that I have a smart phone (and am thus no longer reliant on public or borrowed computers for email access), the vast, vast majority of my email use is centered on a handful of devices, all of which I own. The last time I checked my email on a device other than my own was probably three years ago. Weird, now that I think of it.

The other sacrifice is related to UX. I happen to like Gmail’s interface. In particular, I’ve grown quite used to Gmail’s thoughtful keyboard shortcuts, which make it possible to do nearly navigation without touching the mouse. Coming up with a reasonable facsimile of these shortcuts in Thunderbird would be the biggest part of my configuration process.

Keyboard Shortcuts

Thunderbird has pretty good keyboard shortcuts out of the box. I didn’t feel like learning a whole new system, though, so I wanted a way to map Gmail-style shortcuts onto Thunderbird. There used to be a Thunderbird extension to do just that, called GmailUI. But the GmailUI website suggests that the extension is only compatible with Thunderbird versions 0.8-2.0 (Thunderbird’s currently in 3.1), which would explain why the extension doesn’t show up in a search on tho Mozilla repo (the “Expression Search” plugin does come up, which is a fork of a part of GmailUI that I’ll talk about in a minute – but it doesn’t do keyboard shortcuts).

So I looked for a more general method for customizing Thunderbird’s keyboard shortcuts, and found it with Keyconfig. It’s pretty straightforward to remap keystrokes using Keyconfig (Tools > Keyconfig), though it can be a bit of a pain because many of the standard Gmail shortcuts (like j and k for up/down navigation) are already in use by Thunderbird, so that changing one shortcut often means making two. You might find it helpful to borrow my configuration, which I’ve pasted below. Add these lines to your user.js config file (create if it doesn’t exist):

	user_pref("keyconfig.main.key_killThread", "][I][");
	user_pref("keyconfig.main.key_markJunk", "meta shift][J][");
	user_pref("keyconfig.main.key_markReadByDate", "meta shift][D][");
	user_pref("keyconfig.main.key_markThreadAsRead", "meta shift][R][");
	user_pref("keyconfig.main.key_newMessage", "meta shift][M][");
	user_pref("keyconfig.main.key_newMessage2", "meta shift][N][");
	user_pref("keyconfig.main.key_nextMsg", "][J][");
	user_pref("keyconfig.main.key_previousMsg", "][K][");
	user_pref("keyconfig.main.key_reply", "][R][");
	user_pref("keyconfig.main.key_replyall", "shift][R][");
	user_pref("keyconfig.main.key_toggleMessagePane", "][V][");
	user_pref("keyconfig.main.xxx_key74_SwitchPaneFocus(event);", "][D][");
	

Briefly, this does the following. First, it maps some familiar keystroke combos from Gmail to Thunderbird: j and k to up and down, and r and R to Reply and Reply To All. Second, because two-stroke codes from Gmail (like gi for Go To > Inbox) don’t seem to be supported natively by Thunderbird, I’ve mapped D to SwitchPaneFocus, which lets me get back and forth between the folders pane, the message list pane, and the single message pane, for easier navigation. As I get more comfortable with it, I might write my own extension that ports over some of the other most convenient Gmail shortcuts, but for now this covers a good 80% of what I might regularly use.

The other big shortcut missing from Thunderbird is y, which is the Archive command in Gmail. For that purpose, I installed Nostalgy, which allows you to move messages with keyboard shortcuts. I don’t think I’ve set this up in 100% the right way, but here’s how I’ve approximated Gmail’s y using Nostalgy. First, at Tools > Nostalgy > Keys, I’ve set ‘Save message’ to ‘shift Y’ and ‘Save as suggested’ to ‘Y’. ‘Save as suggested’ seems to work on a session basis; it suggests the folders that you’ve used since the last time you started Thunderbird. Thus, every time I start Thunderbird, the first time I want to archive a message, I use the more verbose shift-Y. A dialog at the bottom of the window suggests places where I might put the message; I select Gmail’s All Mail folder. Then, the next time I want to archive a message, I can use y by itself to go to the suggested (i.e. the last-used) location. Since I just throw all of my email into All Mail – no complex tagging or organization – this is all I need.

Bonus! Offline Access and New Email Throttling

Moving to a non-web-based email client is not all bad. For one thing, moving to a local application means that my email – archives and all – are available offline. I know that Google Gears and some of the new HTML5 goodies mitigate the issue somewhat with respect to Gmail, but offline access for a webapp is always going to be something of a hack.

On a related note, one thing that I have always hated about using Gmail is how it weaves together the process of checking for new email and accessing old email. I like to check for new mail at specified intervals only (once every few hours); anything more than that is extremely distracting. In Gmail, this meant closing the browser tab. Yet, fairly frequently, I find that I need to access an older email from my archive in order to do a specific task. In Gmail, this meant checking my new email as well. Now that I’ve moved to Thunderbird, I can access my archive without checking for new mail – the way it ought to be!

Setting it up this way was not straightforward. There are a few things you’ve got to do:

  1. In Tools > Account Settings > Server Settings, configure the ‘Check for new messages at startup’ and ‘Check for new messages every x minutes’ however you’d like. I have them both disabled, so that email can only be checked manually.
  2. Here’s the tricky part: Gmail, in its futuristic wisdom, uses a special protocol called IDLE to push new email to remote clients – bypassing the settings from step (1). (This took me half a day to figure out.) Disable this feature at Tools > Account Settings > Server Settings > Advanced > ‘Use IDLE command if the server supports it’.

I’m planning to spend a few weeks improving and getting used to this setup before starting the migration to self-hosted email.

Related posts:

  1. Project Reclaim and the email dilemma
  2. Automated and redundant Wordpress backup via email
  3. Making Userthemes work on WordPress 3.0