Recently in Backups and storage Category

I've enjoyed Zack Whittaker's iGeneration blog on ZDNet for a while now, but his most recent entry really hits you where you live — if you have a lot of stuff on a computer hard drive that you'd really not like to lose in an instant.

Zack accidentally kicked out the power plug on his computer, scrambled his hard drive ... and faces a huge, huge bill if he wants to recover years' worth of data:

Through no will of trying, I have now come to a sound, unfortunate conclusion, that my hard drive is well and truly screwed. My data is fully intact but I have absolutely no way of accessing it. And did I back up my machine? Of course I did – but on a separate partition, and on that hard drive.

A backup on the same drive? Bad idea.

That's why we all should ignore the garment-rending in the geek arena over the privacy implications of storing data in the cloud with a service such as Amazon S3 (via something like JungleDisk) or with DropBox, Mozy or any number of competing services.

Yes, there are problems in the cloud and with off-site, networked backups. But the benefits far, far outweigh the risks of not doing multiple backups in multiple ways and at multiple locations.

By all means, follow Zach's advice and back up to a detachable, external hard drive (preferably more than one). But don't think that's enough.

You don't just need a backup. You need a backup plan — one with the kind of redundancy that a cloud backup can add to one that also includes multiple hard drives stored in different places.

I'm getting increasingly comfortable keeping and viewing all of my photos online. Printing them? Yeah, every once in a while, but not every one, multiple times, like we used to.

I've got thousands of e-mails, most of which I probably could lose without tears but which serve as a huge database of information that I'd rather preserve than do without.

I probably should be making printouts of my "important" writing, should I do anything like that at some point in the future. Yep, paper — though bulky — can be the ultimate backup. Clay tablets are good, too.

But the reality is that my aversion to clutter (yes, despite my immersion in it) means keeping more and more things on bigger and bigger hard drives (I just saw a 2TB model from Seagate ... 1.5 TB is old hat, I guess) and making sure, through multiple backups (and backups in the cloud that are worth paying for), that I don't lose all of this data due to my own propensity for kicking out power plugs, or any number of natural and man-made disasters.

As an experiment, I decided to bring my Evolutionary Computing presentation on making the journey into free, open-source software — a slide show originally created in OpenOffice Impress 2.4 — into Google Docs, which happens to have a presentation app in addition to the better-known Docs and Spreadsheets components.

I revised the presentation — taking some things out, adding others and providing some updates on what I'm doing — and output it as a PDF.

Download that PDF for your reading pleasure by clicking on the image above or the link below:

Evolutionary Computing (revised July 2009)

Interesting note: I believe that no previous entry on this blog has been filed under so many categories. (And I've been considering dumping Categories entirely and just using tags ...)

I'm no coding guru. And I feel like having to write my own scripts to get stuff done in Unix/Linux is all too much like reinventing the wheel.

Be that as it may, I hacked together these two short scripts to back up my /home files in Ubuntu 8.04 to an external USB drive. I put the scripts in /usr/local/bin and made them executable. I'm lazy enough that I used the Nautilus file manager to do this.

I run the scripts with sudo, meaning in my user account, I open a terminal and do this, entering my password when prompted:

$ sudo usb-backup

$ sudo usb-backup-exclude

For the second script, I created an "exclude file," which the script uses to exclude whichever directories or files I wish. In this case I use it to exclude the .gvfs directory, which breaks the script (and doesn't need to be copied anyway) and in this case to exclude my Thunderbird mail files, since they take so damn long to back up that doing it every day is something I'm not fond of. The beauty of the exclude file is that I can modify it while keeping the script the same.

I'm sure there are many of you who can do and have done a better job than this, but these two scripts appear to work, and that's what counts for me anyway.

There are some pounded-out notes for the scripts; feel free to remove them. They won't affect how the scripts work.

Here are the scripts:

usb-backup:

#! /bin/bash
# Use rsync to back up the /home folder to a 4 GB USB flash drive
# --delete allows for deletion of files on the backup that have been previously deleted on the source drive
# using --exclude to keep rsync from trying to back up ~/.gvfs
# Finally able to remove --ignore-errors now that .gvfs is excluded
rsync -av --delete --exclude 'home/steven/.gvfs' /home /media/disk/ubuntu
exit 0

usb-backup exclude:

#! /bin/bash
# Use rsync to back up the /home folder to a 4 GB USB flash drive
# --delete allows for deletion of files on the backup that have been previously deleted on the source drive
# setting up an exclude file to back up some directories and not others.
# Finally able to remove --ignore-errors now that .gvfs is excluded
rsync -av --delete --ignore-errors --exclude-from '/home/steven/Documents/shell_scripts/exclude' /home /media/disk/ubuntu
exit 0

And here is my "exclude" file which, as you can see from the script above, lives at /home/steven/Documents/shell_scripts/exclude:

/home/steven/.gvfs
/home/steven/.mozilla-thunderbird

.gvfs note: I've done similar scripts before in OpenBSD and Debian, and I don't believe either used the GNOME Virtual Filesystem, so there was no need to exclude ~/.gvfs when using rsync.

I've been bringing more data into my main Ubuntu 8.04 LTS installation on one of my two Toshiba Satellite 1100-S101 laptops, and I continue to be satisfied with the performance of what by most accounts is the world's most popular desktop Linux distribution.

No, its GNOME desktop isn't as fast as Debian's. But even though I do have Xfce (and not the full Xubuntu) installed on this Ubuntu laptop, I'm still using the brownish-themed GNOME that ships with the distro.

I'm getting used to all the GNOME-ish touches in the Nautilus file manager and in Ubuntu/GNOME in general that makes a full-fledged desktop environment such a nice place to work.

After planning for weeks to take my main production laptop from OpenBSD 4.4 to 4.5, I sweated through the upgrade only to lose what was perfect X compatibility and pull the "kill switch," which in this case was transferring everything in my freshly rsync'd backup to my identical Toshiba Satellite 1100-S101 laptop running Ubuntu 8.04 LTS, a system I've been running for quite awhile on this and another laptop — and which has thus far proven itself to be stable enough for the pounding I give these machines in my daily work.

OpenBSD 4.4 basically "saved" me and one of these marginal Toshiba laptops (both were destined for the garbage) last November when I could barely get an install CD of any type to boot. The install floppy in OpenBSD enabled me to quickly set up a system that worked quite well and did almost everything I needed it to do. And stability was almost a given. I rarely had a problem that wasn't inherent to OpenBSD itself (such as the difficulty of installing Java, nothing past Flash Player 7, the extra steps required to properly configure things such as CUPS).

Since the system ran so well — just like Ubuntu 8.04, video on this Intel-based system ran perfectly with no xorg.conf — I kept it going for the entire six months of the OpenBSD 4.4 release's life.

As those who use OpenBSD know, upgrading the operating system is not as easy as it is in your average Linux distribution. It pretty much comes with the territory that a -release upgrade requires preparation, following instructions, and a bit of manual command-line work. Many times I've heard — both in OpenBSD and in Linux for that matter — that it's easier and cleaner to do a full reinstall rather than an in-place upgrade.

I will still try a full reinstall of OpenBSD 4.5. And I'd like to try running -current — the OpenBSD development branch that can be regularly updated and which is famously stable despite the "development" tag.

But right here, right now, I can't spend weeks diagnosing my X issues (briefly, there's some funky junk hanging from the cursor, and "artifacts" linger on the screen, which isn't redrawn fast enough/often enough to make X usable). The same thing turned me away from Debian Lenny on this and my Gateway Solo 1450 laptop in the months before the then-Testing distro went Stable. Because of my affection for Debian (still one of my very favorite operating systems), I spent weeks trying to diagnose the problem before realizing that dozens of other distros relieved me of the need to obsess (unsuccessfully) over it.

Right now the Gateway, used by our 5-year-old dual-boots Ubuntu 8.04 for her and CentOS 5.3 just because it runs so extremely well on that particular laptop.

And for months now I've had this other Toshiba laptop running Ubuntu 8.04 as a backup. I have Java installed, which I do need. Flash, too. The Opera Web browser.

Today I added Inkscape, Thunderbird, gFTP and Gparted.

On the OpenBSD laptop, I had about 1 GB of e-mail in Thunderbird. It makes rsyncing the box such hell that I'm thinking of writing a script that EXCLUDES the Thunderbird files just so the rest of the backup doesn't take so damn long ... but I digress.

I figured out how to bring my Thunderbird settings and mail over to the Ubuntu machine. I did the same with my Firefox bookmarks.

-- Begin tutorial:

Moving bookmarks from one Firefox 3 installation to another:

• Since Firefox now uses the SQlite database to store/organize its bookmarks, simply moving the bookmarks.html file from one Firefox 3 installation to another will DO ABSOLUTELY NOTHING. You need to do it another way, which I describe right here. First, grab the bookmarks.html file from your old FF installation and put it somewhere in your /home directory where you can easily find it.

• In the Firefox 3.0 installation where you want to IMPORT the bookmarks, go to the Bookmarks tab and click on/choose Organize Bookmarks.

• Click on the Import and Backup drop-down menu and click Import HTML.

• Then navigate to the bookmarks.html file from your old FF 3 installation (you have moved it over already, haven't you?) and click it to bring it into your new installation.

• Note: In Ubuntu at least, this process WON'T allow you to see hidden files or directories, so before you begin, copy your old bookmarks.html file to a place in your home directory where you don't need to go into your old installation's .mozilla directory, for instance.

• FYI: In both of my Firefox 3 installations, the bookmarks.html file is located here:

  /home/username/.mozilla/firefox/xxxxxxxx.default/bookmarks.html

  In the above example, "username" is your actual username, and the eight x's are the unique alphanumeric prefix that Firefox gives to your "default" directory under /.mozilla/firefox/
  

-- End tutorial.

-- Resume rant.

OK, so I'm fully operational in Ubuntu at this point. My respect and admiration for the developers and users of OpenBSD remains, and I hope to get the other Toshiba fully operational under OpenBSD 4.5 as soon as possible.

But I'd be lying if I didn't say I was relieved to have, in Ubuntu, a machine and system that easily updates all of its software with a few clicks and provides me with what — at this point — is a trouble-free working environment.

Of course that could all change. I'll see over the next week how well Ubuntu 8.04 LTS performs on this hardware, with my chosen applications and for the tasks I have.

I could start the distro-hopping merry-go-round and go back to Debian, try out Slackware, ZenWalk, etc., but right now if Linux in this form does what I need it to do (not crash, run acceptably fast, wash, rinse, repeat), I'll be sticking with Ubuntu as long as it fills the bill.