How to create a free off-site email backup storage archive

By Adam Boettiger

[Updated with new information: 4 JULY 2008]

Would it be useful to you to be able to access any email you ever sent or received? What if you could access this power from the palm of your hand via an iPhone or BlackBerry mobile device?

Adam BoettigerQuite a few folks have asked me how I have my email set up.

I can search an archive of over 75,000 email messages dating back a full two years, all from the palm of my hand via my iPhone while on the train commute into work, at the office, on vacation - anywhere.

Sound useful? You bet it is! It means I no longer need to be an Email Pack Rat, accumulating bits and bytes on my laptop’s hard drive, counting the days until my email database file becomes corrupted or crashes. I run a lean ship these days, despite an inbound flow of over 200 email messages per day.

To see if this is right for you, take the following test in your email program:

You corresponded by email with a woman back in June of this year three times about possibly using her virtual assistant services. You can’t locate her phone number and you don’t really remember her full name, only that her first name was “Kristin”. How long will it take you to find the email message from June that has both her cell phone # and email address, allowing you to touch base with her?

It took me less than 15 seconds on any web browser on any computer. That’s powerful.

I feel comfortable deleting anything and everything from my desktop email program, knowing that if I truly ever need it again, it is only but a few keystrokes away. Having an empty Inbox is like breathing fresh, mountain air for the first time after being stuck in smog for months.

In this article I’ll outline the how and why everyone should at least experiement with an off-site email archive, even if they seldom actually use it. My objective in this strategy is to show you how to:

  1. Create a backup of all of your email email (both sent and received, from one or multiple addresses) and to have that backup reside off-site, not specific to any particular machine, external drive or media
  2. Provide for a physical separation between “real” email communications (two-way) and publication subscriptions (inbound-only). It was desirable for me to do this because when I travel I find that I may not want to receive or have time to review the majority of email that comes in, particularly if it is a newsletter or discussion list traffic.
  3. To provide for the ability to control which email is sent to my mobile device (currently an iPhone as of this writing)
  4. To create a highly searchable index of email messages, that does not need to reside on my laptop and take up disk space, but that I can easily access via web browser from any computer, at any location, including a mobile device.

This strategy meets my needs and has done so for some years now. YMMV (Your Mileage May Vary).

NOTE: If you find this article useful, you may want to subscribe to my RSS feed for future articles. My next article here will be about an inexpensive way to reduce spam levels to almost zero in your Inbox.

You may also find a companion article useful after reading this one: “How to block spam and prevent spam email from reaching your Inbox by washing your email“.

PS: A few folks have reached out to me about setting this up for them. I’ve made it as clear as I can here. If it still does not make sense I will be happy to set this up for you for $50 USD (my time) plus actual costs. Contact me at (503) 946-6450 if interested.

Adam Boettiger
adam at adamboettiger.com
(503) 946-6450 voicemail
Portland, Oregon

Short Version - Executive Summary

For advanced users, to save you the time of reading this entire article, here’s what I did:

Long Version - Details

Domains

Depending on your planned use, if you own more than one domain or think that at some point in the coming years you will own more than one domain that you will want to receive email from, it may be worthwhile for you to purchase a “base domain” that all email is routed to, for simplicity purposes.

For example, I own between ten and twenty domains at any given time for various entrepreneurial projects, most with active email addresses that route email to me. Rather than accessing boxes (POP accounts or IMAP) on each, all of them but the base domain are “forwarders” or “aliases”, meaning that they have email addresses that simply route/redirect email to a box on another domain.

The base domain I use is a random three-character domain I picked up some years ago. I never publicize what it is and I never use it online. It is, for all intensive purposes, a “virgin domain”, uncompromised by spammers. The only purpose it serves is to house my base POP accounts/mailboxes. When I set up email in my email client, it is that domain and its mailservers that I use. When I set up a new address on one of my other domains, it is a forwarding address that maps to one of my two base mailboxes on the base domain.

This arrangement serves me well as I can cancel any of my project domains without having to change existing email settings on my email program. For domain acquisition, I use GoDaddy.com, but there are many places to acquire domains.

Hosting

I strongly recommend that anyone who is remotely serious about using email for business pay to have a hosted account outside of their work/employer-provided account. I am currently hosted on a VPS Virtual Private Server at $49/month with Servint. I’ve been really happy with them and been able to reach them by email at any hour, even 1am on a Sunday night, and get a response back within five minutes. They are also quite reachable by phone. This account allows me to host multiple sites and domains and provides for sysadmin level access. A lesser-expensive but good alternative would be a hosted account with Pair Networks. Finally, if you host more than one site or have high traffic and require a larger solution or dedicated server, I recommend Rackspace.com for stellar service and support.

Step 1: Set up a Gmail account for your email archives

Click here and sign up for a free gmail account.

Even if you already have one that you use for sending email, get one solely for the purpose of storing email. Consider this to be your “bucket”, whose only job is to collect email until it becomes full. Mine has 45,000+ messages in it and I’ve only used up half my storage, so it will probably take you a while to come close to needing to open a second account. Years perhaps.

For a username, first check if firstname.lastname@gmail.com is available. i.e. John.Smith@gmail.com. It’s always good to at least own your name. However if someone else has taken it or there are others of you out there with the same name, consider using something like your last name and the word “archives” or if you prefer, “bucket”, like: “smith.archives@gmail.com” or “johnsmith.archives@gmail.com” or “tim.bucket@gmail.com”. This will be your off-site, off-machine storage area that you will search via your mobile device or any computer with a web browser.

In the event that your primary computer is lost, stolen, has a meltdown or burns in a fire, this “bucket” will also be what allows you to simply jog down to the library at a public computer terminal and pick up where you left off, without missing a beat or message, waiting for your new machine to arrive.

Step 2: If you own a mobile device, set a bookmark in your mobile browser

mobile.google.com or google.com/gmail*

*Note: The mobile version of Google Gmail (above left) is not nearly as robust as the full version. If you are using an iPhone, you have a real web browser, none of this WAP crap. Go ahead and bookmark the full version (above, right).

Step 3: Set up Public and Private POP accounts

Your web hosting service control panel should allow you to create POP accounts (mailboxes). Create one that will be used for private email exchanges and a second one for one-way, inbound lists, newsletters, software registrations etc. Suggested configurations are:

private@yourdomain.com
public@yourdomain.com

NOTE: Throughout this article I will refer to “yourdomain.com”. This is a fabricated placeholder for whatever domain you ultimately decide to register and use.

Step 4: Create a private alias that forwards to BOTH your new private mailbox and your new email archive

If your first name is John or Jane, consider the following configuration for creating a private email address:

john@yourdomain.com >> private@yourdomain.com, john.archive@gmail.com
jane@yourdomain.com >> private@yourdomain.com, jane.archive@gmail.com

NOTE: The above illustrates an email forwarding address on the left and the two “boxes” that the address forwards or “maps” to. In the above example inbound email sent to ‘john@yourdomain.com’ will route to both your private mailbox and your Gmail archive box, the idea being that you can delete email at will, without fear that you will need something later on. If you do, all of your email will be available to search in your archive account. Keeping your inbox empty and your email database file small will give you peace of mind and reduce the risk that your email file will become corrupted due to large size.

Step 5: Set up a public email forwarder

In creating a public address to use for subscribing to newsletters and other publications (non-correspondence, inbound-only mail), I chose to use a service called POBox.com. This is an email forwarding service that effectively forwards all inbound mail through their servers to one or multiple boxes or addresses that you specify. The service runs me around $20/year. My rationale for using the service is due to their strong server-side spam filters that allow me to control or block inbound messages by country of origin, scrub against ISP blacklists and more. It is quite a robust service with good filters. I use a pobox.com address as my primary email for inbound-only mail traffic, such as newsletters, email discussion list participation, software registrations and anything that is not 1-on-1 correspondence-related.

Yes! I do realize that Gmail has superb spam filters. The reason I recommend the above forwarding service is because Gmail’s spam filters are not customizable. They are either ON, or they are ON. With a POBox account you’ll have the option of filtering inbound email against all the major DNSBL (blacklists) of the major ISPs in addition to being able to block email by country of origin - things that Gmail simply cannot do.

If your name is John Smith, suggested configuration is as followed:

john.smith@pobox.com >> public@yourdomain.com, john.smith@gmail.com

NOTE: If you choose not to use the POBox.com service, another option would be to create a public forwarding address on your own hosting service. Suggested configuration might be something like this:

john.smith@yourdomain.com >> public@yourdomain.com, john.smith@gmail.com

Step 6: Set up your public and private POP accounts in your email client of choice

Whether you use Entourage, Thunderbird, Mail or Outlook, follow your hosting account instructions and add your public and private mailboxes to your email program. It is beyond the scope of this article for me to describe how to do this and unnecessary as instructions are provided by all hosting providers. If you have difficulty, contact your hosting provider for assistance.

Step 7: Test your new addresses and accounts

Before making suggested modifications to your Gmail account below, you’ll want to test your new addresses to ensure they are working correctly. If you just purchased a new domain name, understand that it can take up to 72 hours for the nameservers to propagate. In English, this means that if you’re using a new domain name and your addresses don’t work, bookmark this page or print it, wait a few days and then try testing them again.

To test your Private Address:

To test your Public email address:

Step 8: Gmail Mods

Here are some tips that can help you make the most of your new Gmail archive. Adopt what you wish.

Add “From” addresses to your Gmail account

Make a list of all of the email addresses that you send email FROM and in Gmail Settings area, add these addresses to the account to allow you to send email from those addresses. You’ll be sent a confirmation message from Gmail for each address you add to verify that you are the account holder. Some email discussion lists require that email be sent from the subscribed address, so adding these addresses will allow you to participate in discussions or send email while you are looking at your archives if you need to.

Settings >> Accounts >> Add another email address

For example, you may at some point need to send email from your Gmail archive from either your private or public email addresses, so it makes sense to add: john@somedomain.com as well as john.smith@pobox.com. That way when you send email from your archive account any replies will be sent to your desktop.

Use a filter and a label to organize your list email

There will be times when it will be convenient for you to be able to view all messages sent to your private email address. There may also be times when you want to instantly see all messages from a certain discussion list. You can do this with one click if you set up a filter and use a label in your Gmail archive.

Settings > Filters > Create a new filter > To: john@somedomain.com > Next Step > Skip the inbox (archive it) > Add the label: john@somedomain.com

What you’ve just done is to tell Gmail that all messages that arrive in your Gmail account that are addressed to your private email address, john@somedomain.com, should be automatically archived (not left in the Inbox) and that a label of ‘john@somedomain.com’ should be applied to each.

After you create this filter, on the left side of your account you’ll see a hyperlink ‘john@somedomain.com’. From this point forward, anytime you wish to view only messages sent to your private email address, all you need to do is click the label on the left side. Very cool. And useful.

Another example might be if you received email daily from a discussion list. Let’s say iPhone-L, the iPhone Email Discussion List. Take a look at any message received from the list and note something common about each message. For example, on this particular discussion list, the To: header is always iphone-l@iphone-l.com, and there is always the following in the subject line of each message: [iPhone-L].

Go into your Gmail archive and create a filter for this list:

Settings > Filters > Create a new filter > Subject contains: [iPhone-L] > Skip the inbox (archive immediately) > Add a label > List: iPhone-L

What you’ve just done is to tell Gmail that for every message from this discussion list that arrives in your archive, apply the label ‘List:iPhone-L’ and then archive it. After you create this filter/label combo you’ll see a new hyperlink on the left side of your account (List: iPhone-L). You can effectively do this for any newsletter or discussion list that you may have a need to see only posts from that list at once, with one click. Perhaps you’re in the library or on a business trip overseas and want to read only that list. You can do so from any web browser.

Why ‘List: iPhone-L’ and not just ‘iPhone-L’?

Of course you can label any way you wish. If you preface your lists with ‘List:’, then they will all be grouped together on the left side, thus giving you the appearance of being more organized. ;)

While you’re at it, create a filter and label for ‘john.smith@pobox.com’ so you can have one-click access to all your public messages if you ever want it.

Consider archiving your work email

This is a gray area, but one I feel is worth mentioning.

Say you work at Dunder-Mifflin Paper Company and your work email address is ‘john.smith@dunder-mifflin.com’.

Go into your web hosting account and create a POP/mailbox called ‘work@yourdomain.com’.

Now create an alias/forwarder that routes as follows:

blackberry@somedomain.com >> work@yourdomain.com, john.archive@gmail.com

What you’ve just done is created a third mailbox to physically separate work email from personal email. You’ve also created a forwarding email address that, when sent to, sends a copy of all email to your new mailbox as well as your Gmail archive.

Next go to your IS guru or company system administrator. Tell them that you want to get work email wirelessly on your phone and that you’ve created an alias to do so. Ask them if they can set it up such that in addition to your work computer, any email received at john.smith@dunder-mifflin.com also forward a copy to blackberry@somedomain.com.

If they balk at your request, be prepared to bribe them with caffeine - a $20 gift card to Starbucks works well, as does a six-pack of JOLT Cola. That should buy you about 30 minutes of their time.

If they kindly offer to set up your wireless device to access your account via the company’s exchange server, you can accept it, but I’d politely decline, telling them that you’d rather keep things simple and just have a copy of your mail forwarded to a single address rather than messing with trying to get exchange working on a wireless device.

(Yes, I know there are advantages to having exchange access to email on your blackberry…but that would defeat the purpose of what I am showing you.)

Ultimately they set it up for you. So now all inbound email to your work address is also copied to your own POP account and it’s also forwarded to your new Gmail archive.

Now go into your Gmail account, and create a filter that looks for john.smith@dunder-mifflin.com in the To: header, archives it immediately skipping the inbox, and applies the new label ‘john.smith@dunder-mifflin.com’.

Now when you’re at any web browser, anywhere in the world, and you want to check office email, just log into your gmail archive, click the label link and you’ll see all of it.

Additionally, because you used an ALIAS, and did not take the IS guy up on his offer to set up exchange on your wireless device, you can now add new addresses to forward to or change from a blackberry to an iPhone if you want to, without having to go back and involve the IS guy and spend more money on more gift cards or caffeine bribes.

The last thing to do is set up this third POP account in your desktop email program on your personal computer and/or set it up as an IMAP account on your blackberry or iPhone. This is optional of course, but it assumes that at some point you may want to check your work email while out of the office without having to lug your computer home with you.

Now - with your work email also being stored collectively in your new gmail archive, you have cleared the way for you to effectively become the smartest person in the company. Need to reference that email from months ago about XYZ? Sure! Tap tap tap. Wow that was fast. Now you can feel free to delete at will from your laptop at work, to keep your mail file small, knowing that you can get anything you want to at any time through your archive.

Use filters to keep your Archive Inbox empty automatically

If you do not create this filter, all copies of mail sent to your Gmail archive will build up in the Inbox. You may want to create a filter that takes all inbound email and skips the Inbox, archiving it. When you view your account it will appear to have an empty Inbox. If you want to view messages you’ll either need to search using the search box or click the All Messages link on the left side to view them. Otherwise, even with 45,000 email messages in your archive, the inbox will appear empty.

To keep your archive inbox empty, create a filter in Gmail that looks for the “@” sign in the TO header, then check the box that says “Archive immediately, skipping Inbox”. If you need to see mail later on you can either click one of your labels or you can click the link that says Show All Email.

Why Gmail?

In my opinion, Gmail has the best search interface of any of the web email solutions. Because you set up an off-site archive, you can access it from any computer with a web browser and use Gmail’s Advanced Search feature to search by Label and keyword or subject to quickly find any message you received, dating back to the day that you started your archive. The storage space on Gmail is ample enough such that even the most power email users will find that a single account should last for at least two to three years of archived messages.

“What if I run out of space in my Gmail account?”

Gmail currently has a cap of around 2.9 GB of storage on their free accounts. For power email users that should last about two years or 60,000 email messages, depending of course on message size and attachments. When you get close to your cap Google has the option of upgrading your storage capacity for a fee, starting at $20 per year, which is reasonable. So no need to worry about running out of storage.

Keep in mind, however, that Gmail may have a period of time when they may deactivate your account if there has been no activity in some time. I believe it is nine months. This means that in a few years, should you wish to keep your very old email on accounts that are full, you may need to set a tickler to log into that account at least once every six months to keep it active.

There is a superb service that can help with this. It’s called MyTicklerFile.com and will allow you to set ticklers in the future that will notify you by SMS text message and/or email at future dates and times. Definitely worth exploring!

It’s also a good idea to maintain a list of any gmail accounts, aliases and POP accounts that you have by creating a Word document or Excel sheet and saving it somewhere safe to refer to from time to time as you re-evaluate your email strategy.

How to address the issue of archiving SENT email

The system above allows you to archive every single inbound email message you receive on multiple accounts. This is useful not only in the event that you need to locate something quickly that someone sent to you (if you were on the phone and quickly needed to locate it), but also in the event that your computer is stolen or you experience a hard drive crash. Should this happen your down time will be minimal (all you need do is go to another computer and access your archive via the web and start working again) and no data will have been lost. The system above is also advantageous because it means that you can feel comfortable deleting anything and everything from your email programs that normally store messages on your hard drive. Keeping your email program lean and free of thousands of messages will provide for a faster running program that uses less memory to operate, with less chance files will become corrupted.

However, in many instances you may want to locate a message that you SENT to someone, not just received, perhaps to reforward it to them or follow up on something you sent. Most email clients now have a feature that allows one to BCC an address on all outbound email. In Thunderbird this is shown on the account preferences area. Simply go into the preferences of your email program and enter in your off-site archive address (john.oss@gmail.com) into the BCC area. Once you do this, provided that you do it on all email clients that you use, your archives should then hold a record of all inbound and outbound email, providing for a complete archive that is both highly searchable and not associated with any particular medium or hard drive.

If Outlook is your email program, under Tools you can create a rule that filters OUTBOUND or SENT email messages, such that you can CC yourself. Setting a filter to CC yourself on sent messages and a second filter to look for yourself in the From: header and delete those messages when they come in will allow your Gmail archive to have a full record of both inbound and outbound email. I cannot speak for other email clients. Try them to see if your favorite is able to do this.

Premium Options with Google Apps Premier

Google Apps Premier is the fee-based upgrade now available for those who wish to pay $50 per year for some powerful features not available in the free versions of Gmail, Google Calendar, Google Sites and other apps in the suite of G-Products. This link provides a page that compares the feature set between the free and fee-based versions. Where Gmail is concerned, which is the purpose of this article, the primary benefits of using the fee-based account as your Gmail archives are that it provides 25GB of off-site storage for email rather than the initial 5 or 6 that Gmail Free provides.

I personally find tremendous value in using Google Apps Premier and the Gmail client for my own email. It is well-worth the fifty buck investment and quite convenient. More powerful than the free version and no advertisements. You can sign up for a 30-day free trial of Google Apps Premier through the link above. It does require credit card, but they charge it only if you do not cancel within 30 days.

A word of warning to those of you who do not currently manage your own domain names: You may want to stick with the free version as the Premier version of Google Apps requires the use of a domain name. If you’re somewhat knowledgeable about things Web-related you can probably figure it out, and there are also companies that can help you set it up and redirect MX records; but know that it is more involved than the free version of the suite.

I’ve been running the email archiving strategy I describe in this article now for over two years, with some 90,000 email messages archived in my searchable index. I’m still only at about 60% of my storage cap on the free version of Gmail. However I do have plans when I hit capacity on the free account to pony up for the fee-based version of Google Apps as a long-term archive solution for email.

Migrating Email from One Gmail Account to Another

Once you hit capacity with the free version of Gmail, you’re faced with a problem if you want to move to the Premier version of Google Apps Gmail: How does one migrate 70,000 email messages to a new and larger-capacity account?

Actually, it’s not as difficult at you might think, but it does take some time. Gmail allows you to use POP to import email from other accounts. So what you want to do is something akin to the following:

  1. Set up your Google Apps Premier account, ensure the domain and MX records are propagating, test it for receipt of email.
  2. Log out of your Premier account and log into your old free Gmail archive account. Under the Settings link, click to turn on POP mail. This allows you to use other applications to access the email in your old archive account via POP.
  3. Save your settings and log out. Then log into your Premier apps account again - or perhaps just consider using Firefox 3 as a browser and opening two tabs so you can switch back and forth between the two.
  4. Under Settings > Accounts > Get email from other accounts, add your OLD Gmail archive account here.

Once your Premium apps archive account has this information it will begin to import your 70,000+ email messages from your free Gmail account. This will likely take a few days for it to do, but it will happen. There is a fairly good article in the Google Apps blog on how to do this here.

Dotting the i’s and crossing the t’s

When upgrading from your free Gmail archive to a larger-capacity fee-based one ($50/year), don’t forget that your new archive email address is different from your old free one. This means that you’ll need to revisit all the filters and aliases that you had been routing email to and change the archive address from the old free account to the new one. The easiest way to do this is to make a list of all of your domains and email aliases used on a piece of paper with a pen and just check them off as you make mods to the aliases/routing.

You are likely moving to a higher-capacity, fee-based Gmail account because your free account has capped out for storage. If this is the case, you may also want to go into your new Premier account and under the new POP you just added that is importing from your old archive, you may want to specify that once it has imported your messages it removes them from your old archive account (remove from the server), rather than leaves copies of the email.

Once all the messages in your old archive account are migrated over, you should have zero messages in the old account. At that point it will be quite easy for you to tell if you’ve remembered to redirect all existing aliases to the new archive account or not. You should only be receiving email at the new archive account and should receive no email at all on your old archive account. If you are still receiving email on the old archive account, track down which alias or account is still forwarding to it and make the adjustment to reroute to the new archives.

Feedback/Questions/Additions/Corrections?

Reach the author at adam (at) adamboettiger.com or provide your feedback by voicemail at (503) 946-6450.

Was this valuable to you? Send a donation to the author to let them know!


Suggested amounts:

NOTE: If you find this article useful, you may want to subscribe to my RSS feed for future articles. My next article here will be about an inexpensive way to reduce spam levels to almost zero in your Inbox.

You may also find a companion article useful after reading this one: “How to block spam and prevent spam email from reaching your Inbox by washing your email“.

AB

September 29, 2007 | Filed Under Email 

Comments

Leave a Reply

You must be logged in to post a comment.