Recently in A Month on the Command Line Category

After a few weeks of wrangling with e-mail at the console, I now have a working setup that allows me to send and receive mail from two accounts, one personal, the other for work, on my Debian Etch box.

(For those who haven't been following that closely, my hardware is a Maxspeed Maxterm thin client with hard drive and CD-R drive connected to the single IDE header. The motherboard is by ECS, it's sort-of mini-ITX sized, the processor is a VIA C3 Samuel at 1 GHz with VIA's PC133 chipset, and memory is maxed out at 256 MB.)

I'm using mutt as the mail client and msmtp as the SMTP client for outgoing mail. And while real geeks use vi, I've become partial to nano of late, and that's been my default editor for e-mail ... and everything else.

My personal account is with freelinuxemail.com, which looks like, acts like and is provided by fastmail.fm, but which includes free use of their SMTP server. Linspire sponsors the service, and I think they're picking up the tab. (Thanks, Linspire!) The service is IMAP only, which I prefer anyway, because I need to access my mail from a variety of places, programs and PCs.

However, my work account is on a POP server, and I really don't want to pull all the mail off the server and onto a single PC -- and I could never get the company's outgoing SMTP server to work. Instead, I'm using the SMTP server from my broadband provider, DSL Extreme (who's tech support department declined to help me with this project, since, as they told me, "You know we don't support Linux." I figured it out anyway, and I will keep trying to figure out my company's SMTP server, even though everything is working just fine like it is.

The main sites that helped me get all of this working were the main mutt site (mutt.org), the msmtp site, an article from O'Reilly's MacDevCenter.com titled How to Use mutt, FastMail, and Mail.app Together on Your Mac, even though I'm not using a Mac, and My First Mutt, and all are required reading before you embark on this yourself. That and about a hundred Google searches for sample configuration files and advice specific to these programs and my mail providers.

And before I continue, yes, I tried Pine, but I could never really get it to work, especially for the outgoing mail.

The key here is that mutt is configured to read e-mail from my IMAP account. To read my POP mail, I hit lower-case "c" to navigate to other folders, and instead of an IMAP or local folder, I go to my POP site and then log in. The e-mail comes up from my POP account, and I can read, delete and reply to messages, but what I don't delete remains on the POP server, accessible by my Web-based client, Outlook on my Windows box and Thunderbird (or SeaMonkey) on my Linux boxes. (I got the lower-case "c" trick from My First Mutt. Works like buttah.

Here's how I do it:

(type lower case letter) c
pop://pop.mypopaddress.com (note: use pops:// if it's a secure server)
(at prompt for login, enter my e-mail address)
(at prompt for password, enter my password, then the mail appears)

As I said, I haven't yet gotten msmtp to work with my company's SMTP server yet, but I did get it to work with DSL Extreme's (substituting my work e-mail address in the "From" category).

I can choose to send mail from three different accounts, freelinuxemail.com, my work account and dslextreme.com with key bindings I created with help from msmtp's manual (the section on using msmtp with mutt).

Things I'm going to work on in the near future are shortcuts for frequently-used e-mail addresses and some kind of system for organizing saved mail in my /home directory.

Here are my config files:

.muttrc

set use_from=yes
set editor=nano
set imap_user=myaddress@freelinuxemail.com
set imap_pass=mypassword
set spoolfile=imaps://mail.messagingengine.com/INBOX
set folder=imaps://mail.messagingengine.com/INBOX
set record=+"Sent Items"
set postponed=+"Drafts"
set certificate_file=~mutt_certificates
set envelope_from=yes
set sendmail="/usr/bin/msmtp"
set realname="Steven Rosenberg"
macro generic "<esc>1" ":set
from=myaddress@freelinuxemail.com"
macro generic "<esc>2" ":set
from=myaddress@dailynews.com"

.msmtprc (make sure this is set to read/write ONLY by you. The command is: $ chmod go= .msmtprc )

account linux
host mail.messagingengine.com
from myaddress@freelinuxemail.com
auth on
tls on
user myaddress@freelinuxemail.com
password mypassword

account dsl
host smtp.dslextreme.com
from myaddress@dailynews.com
#for "user" in DSL Extreme, the @ and everything after it are not needed
user mydslextremelogin
password mypassword
#auth login is the only authentication that worked (I went through every one that msmtp offers before I tried this one)
auth login

It's probably not a great idea to have passwords in the config files, but it does speed things up, and once I pull the CD-ROM drive, it'll be pretty secure, especially because it doesn't boot from USB either. That's assuming anybody around here a) wants to read my terribly exciting e-mail and b) knows what Linux even is, let alone how to hack into it.

One of the things I'm going to try in the future is to remove the passwords and maybe even the user names and see how the setup works then. For now, it really speeds things up to have user names and passwords entered automatically.

Another must: getting the e-mails to display with the newest on top (they're now on the bottom).

It's taken a couple of weeks to get to this point, and I'd like to thank all the readers of this blog who wrote comments and told me about their console-based e-mail setups.

While on this project, I also set up all of my mail accounts in Thunderbird, but where's the fun in that?

Again, the greatest thing about e-mail at the Linux console is the blinding speed with which e-mail can be read, replied to and deleted.

I've been keeping mutt running on tty1, with a text-based Web browser -- either lynx or elinks -- on tty2. Both browser have their good points -- elinks has better CSS support, but sometimes looking at a page with no CSS in lynx is better. I'm sure I can turn off CSS in elinks, but it's just as easy to start the other app.
I should probably start using a third virtual console with Midnight Commander running, but I haven't been using mc that much lately. Still, if you're using the Linux command line and not using at least two or three virtual consoles, switching between them with alt-F1/F2/F3/etc., you're not really living.

As far as the e-mail setup goes, there are probably better ways to do it, and like all highly configurable applications at the console, it has the potential to grow, change and morph depending on my needs and skills.

While I cited many of the Web pages that helped me, I'd also like to acknowledge the books that have taught me much about the Linux command line. My main book, my bible if you will, is "The Linux Cookbook, Second Edition" by Michael Stutz (here's the free online version of the first edition, but it's well worth buying and having at hand). Stutz's book is so well-written and complete that it's an enjoyable read as well as a valuable reference. Even though it was published in 2004, it remains extremely useful, since the command line isn't subject to the faddishness of the GUI ... not that a new edition wouldn't be welcome.

I've also been using "Linux in a Nutshell," by Ellen Siever, Stephen Figgins and Aaron Webber, another great O'Reilly book, and I've just gotten the other Linux cookbook, titled "Linux Cookbook," by Carla Schroder, also from O'Reilly ... also published in 2004 (and still plenty useful, although again, a new edition would be great.

Since this is Day 27 of A Month on the Command Line, here are the things I haven't gotten done yet:

-- Printing: I tried the trusty Apsfilter, which has worked for me on occasion in Damn Small Linux, but I couldn't get it to find a network printer. I will try CUPS before this is all done, and I think that will work. But why didn't Debian ask me about printers during the install?

-- vi/Vim: I never got that good at running vi or Vim. I read somewhere that even when you're running vi in Debian, it's really Vim. I'll have to look into that. But while I type fast and have to write stuff really fast, I didn't crave the ease with which I could move around the page in vi. I grew attached very quickly to nano, which, with it's F3 keystroke to save and F9 to delete a line, quickly became my favorite command-line app.

-- Images without X: I didn't do much with Imagemagick. Since I couldn't actually view an image in that suite without X, I never got to the point of sizing a photo at the command line. Zgv was already installed in Debian, but it died while trying to display a photo. I checked the config file, and it should've worked but didn't.

-- Blogging from the command line: I never did try out the program that allows direct posting to Movable Type blogs. It's something I'd still like to try, but once I had e-mail working, the will to continue wasn't quite as strong. I did post from elinks to the blog, but it wasn't easy (and for some reason the posts didn't go live without "intervention" from a GUI browser).

But I did a lot of Web browsing with lynx and elinks -- and both are extremely fast and enjoyable ways to read text on Web pages. I highly recommend trying them at the command line.

What separates Linux from Windows and Mac is the fact that Linux from the command line is made up of apps that are up to date (they have to be, given that they're running production servers at very high levels in the enterprise) and extremely configurable. The fact that you can run the Linux command line on hardware that's extremely old and underpowered, yet get all the modern security features that Linux has to offer, is a powerful incentive to use it, especially for older hardware. I've blogged extensively on how to keep older Macs running, on which browsers work, which e-mail programs support today's servers (answers are few and Netscape Mail) and how a decade-old, unsupported OS can function in the modern computing world. But with Linux, all that fades away. Window managers go from full-featured and heavy to extremely light ... and there's always the shell. Linux developers care about users -- and not about rendering old hardware obsolete so you'll buy whatever's new just so you'll have apps that run.

And while the books I've mentioned above have been very, very helpful, I'd love to see a command-line book that walked the newbie through all the apps and setup of a Linux system, with specific instructions on how to set up e-mail, network connections, with even more information on all the current command-line apps. Kind of like what the "Cookbooks" do, but with more specific information on going from nothing to a full command line system. I guess there's not quite the market that there is for Ubuntu/SUSE/Fedora books, or for books aimed at sysadmins, but it would be nice to see. The fact that you can easily set up even Ubuntu as a command-line-only system -- and not just as a server -- makes this something that I think more people would do, if they only knew how.

With the help of My First Mutt, I figured out how to import POP mail into mutt:

To browse a POP3 mailbox, just hit c to change mailboxes. Then, instead of typing the name of a local mailbox, you can enter the location of your POP3 server. This is much like typing the URL for a page in a web-browser:

pop://username@mail.example.com/

Mutt will ask you for your password and you can browse, delete, edit and save the messages as normal. If your POP3 mail account supports SSL encryption, then you should use pops:// instead of pop:// to enable it.

And it works for outgoing mail, too! I guess it's a lot to type, and there's probably a way to handle it through .muttrc, but getting ANYTHING to work is a giant step for me, and I'll just bask in the glow until tomorrow.

Again, I don't know how this works, but I can seemingly read the mail in mutt while it remains on the POP server, which is exactly how I want it for the time being. A total accident -- but freakin' brilliant.

I will post configuration files at some point in the near future. (Trust me, there can never be too many configuration files on the Web, even from one as inexperienced as myself).

To recap the programs I'm using, mutt brings in the mail from an IMAP server (and now from a separate POP server), and that mail is sent out via msmtp. I had some success with fetchmail and pine for the POP account, but I still can't get pine to send any mail. At some point I'll probably need to throw procmail into the mix.

The IMAP service I'm using is freelinuxemail.com (from the company that also offers fastmail.fm, except that freelinuxemail.com includes free use of an outgoing SMTP server. And for those who care about it, the identical Web interfaces for freelinuxemail.com and fastmail.fm are extremely quick to navigate and also very light on resources (with the option of turning CSS stylesheets off). And fastmail.fm was the first free e-mail service I've seen that promotes the use of IMAP, where the mail remains on the server and can be accessed by any number of mail clients from multiple locations -- instead of POP, which generally downloads all messages to the client computer, leaving you out of luck if you want to change mail programs or locations. For me, I'm reading e-mail in three different places on up to five different computers, so IMAP fits me a whole lot better.

For the past few years, though, I've been using Web-based mail services (chiefly Yahoo! Mail and our company's Web mail interface (which, ironically, is like having IMAP service, although for external mail clients we only have a POP server). But for the speed alone, it's nice to get away from Web interfaces and back to standalone mail clients.

Again, and for the fifth or so time, IT SHOULDN'T BE THIS HARD.

I've had some success with my IMAP mail from freelinuxemail.com. Mutt and msmtp are handling things very well. I'd love to beef up my .muttrc file to sort things (maybe I need procmail) and I'd like to figure out how to create additional folders on the IMAP server, but for now just getting e-mail and being able to send it is good enough.

As I said in a comment on the last A Month at the Command Line post, I tried pine again.

Well, first I tried to piggyback a POP account on my current mutt/msmtp setup, but that wasn't working at all. So I tried to configure pine as a POP mail client. After I tried the settings in about a half-dozen Web tutorials, I was able to download all my POP mail (which unfortunately meant that those e-mails are no longer on the POP server and are no longer accessible by the company's Web-mail interface ... and no, there doesn't seem to be IMAP capability).

I've got to tell you, in this day and age of always-on Internet, IMAP is the way to go -- you can read and write e-mail from a variety of locations and computers and not have mail stored on one (unless you want to save them locally, that is). It makes for a lot more flexibility.

But back to pine. I used fetchmail to get the mail into the box and put it wherever mail is supposed to go (into my Linux user account) so mutt can find it. I'm not rock solid on the particulars, but it works. Sending the mail -- that's where I have a problem. In mutt, I needed msmtp (or esmtp) to send mail. Pine doesn't seem to need an external program to send mail, but I just can't get the server to accept my login and password. There might be authentication, SSL or TLS that I need to address. I've got to look into it again and then figure out how to do it in pine.

But I'd rather not use pine at all and do everything through mutt ...

I did spend a little time in Xubuntu 7.04 today ... during which Firefox crashed ... then IE 6 crashed under Wine, so I rebooted into Debian's console and began using lynx and elinks. I don't know if it's frames or java, but elinks seems to handle things better -- at least at dailynews.com. It makes reading text on the Web a whole lot quicker -- and might be worth using even in a GUI.

Anyway, while reading the Daily News story by Eugene Tong about a lady who's been to 177 straight "Tonight Show" tapings, I clicked on an HTML "mailto" link for Eugene and was immediately in mutt, ready to compose an e-mail. It all happened so fast and worked so well.

Yep, I'm in Day 2 of having mutt and msmtp doing my e-mail, and it's been really great.

And while I'm on the topic of e-mail at the Linux command line, I got a nice comment from console user kotnick on how he handles e-mail:

Well, wouldn't this be easier (this is my own setup):

1) fetchmail to get mail from all accounts which provide pop3/imap (I get mails from 6 different accounts)

2) procmail to sort it out and filter out spam with spamassasin

3) mutt to read mail (although I use claws-mail)

4) sendmail to send mail (you can use it on every smtp server that provides relaying, as gmail does, as my local isp does)

4a) if you find sendmail too cryptic, you can try postfix

And from my previous e-mail entry, reader jason does this:

I use Fetchmail, Mutt, Procmail and Postfix to handle my mail, and have also changed my mind several times over the last few years over which anti-spam tool to use.

Here's a brief summary of how it works:

Fetchmail retrieves incoming messages from my accounts on IMAP servers, and delivers them via SMTP (port 25) to Postfix. Fetchmail configuration involves creating a ~/.fetchmailrc file.

Postfix queues the messages and invokes Procmail to deliver them . mailbox_command = procmail -a "$EXTENSION" in /etc/postfix/main.cf

If there is a ~/.procmailrc file, Procmail applies the rules in it to filter the incoming messages into various folders. This implies that you don't need a ~/.procmailrc file for this setup to work, but by having one you can filter out spam and duplicate messages, filter various mailing lists into different folders, etc.

Mutt accesses the local mail folders (/var/mail/username as the primary inbox, and folders under ~/mail) to read the mail. Of course, mail is delivered and filtered entirely in the background.

For outgoing messages, you can configure Postfix with the relayhost command, as in relayhost=mail.isp.domain to deliver outbound e-mail via your ISP's SMTP server. Alternatively, you can just let Postfix deliver mail to the destination directly, without any additional configuration required. Some spam filtering services may filter out your messages if you do this. In my case, I can't tell the difference between the two alternatives, so I just let Postfix handle outgoing messages itself (without relaying via my ISP).

The point to emphasize here is that all of this isn't too difficult to set up (a certain investment in reading manual pages and setting up configuration files is needed, especially for Procmail and to a lesser extent Fetchmail), but once set up, it works really well - year after year - with only incremental improvements. That's why some of us are so addicted to this way of working. I can't remember how long it took to set up the first time, but given how well it has worked ever since, it was well worth it.

After days and days of being able to receive IMAP mail but not send it with mutt, I finally cracked the problem. What took me a little while to understand -- that mutt needed a separate SMTP client to send Internet e-mail -- took a lot longer to actually get working.

First things first: The IMAP mail service I'm using is freelinuxemail.com, which I believe is sponsored by Linspire. The mail service is run by fastmail.fm, but what makes freelinuxemail.com different is that you get to use their outgoing server for free; it costs extra with the plain fastmail.fm.

At the freelinuxemail.com/fastmail.fm site, there are configuration instructions for pine. (I tried them, they didn't work, deleted pine, days later reinstalled pine, tried again, still didn't work). But nothing for mutt. Some of those who replied to my previous post are using up to four separate applications to handle their e-mail, and I was getting mighty dizzy from the whole thing. I tinkered with my configuration files for days, but still nothing).

I thought the problem was esmtp, the SMTP client I was using.

While pursuing other solutions to the outgoing-mail problem, I looked around for other SMTP clients and found msmtp.

I immediately knew what I was doing wrong.

From the instructions for using msmtp with mutt (bolding for emphasis added by me):

2. Configure msmtp:

Create the file .msmtprc in your home directory, with no more
permissions than user read/write (0600, -rw-------).

I had the wrong permissions on .esmtprc all along. How did I miss that?

To change the permissions on the .msmtprc configuration file, I turned to Michael Stutz's "The Linux Cookbook, 2nd Edition," Page 169, for the command:

$ chmod go= .msmtprc

Now .msmtprc was set to private (read/write only by me), and I was in business.

I'm pretty sure resetting the permissions on .esmtprc would have yielded the same result, but for now, I'm sticking with msmtp, and I'll try to figure out how to use this combination (mutt and msmtp) to access a POP account, and, hopefully, multiple IMAP and POP accounts simultaneously.

By the way, while it did lead me astray at times, the instructions I used for initial configuration of mutt and esmtp were from O'Reilly's MacDevCenter (although the path to esmtp was different, I didn't need to match anything up with OS X's Mail.app since this is Linux, and I guess on OS X the read-write permissions of .esmtprc were not an issue).

Another good mutt resource on the Web is My First Mutt, which I'm just beginning to explore.

I will post my config files for mutt, msmtp and esmtp as soon as I get the time.

And thanks to all who helped and encouraged me to continue with e-mail on the command line.