Smtp auth

From Finninday
Revision as of 03:25, 24 November 2010 by Ezekypy (Talk | contribs)

Jump to: navigation, search

Postfix config for smtp auth

Here is an Ubuntu-specific recipe that looks like it will do the job:

https://help.ubuntu.com/7.04/server/C/postfix.html

Currently, my mail service is working as long as I don't try to send mail from a remote machine. For instance, if I have a laptop configured to send mail outgoing mail to my server and am connecting through an untrusted network in a coffee shop or a friend's house, I am unable to connect to the server. This must be fixed.

There are several differences between my existing /etc/postfix/main.cf config and the recipe linked above:

Current Proposed
smtpd_sasl2_auth_enable = yes smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain = finninday.net smtpd_sasl_local_domain =
broken_sasl_auth_clients = yes
smtp_use_tls = yes
smtp_tls_note_starttls_offer = yes


However, my server currently generates the correct list of available services when starting a transaction:

<pre> root@weasel:/etc/default# telnet localhost 25 Trying 127.0.0.1... Connected to localhost.localdomain. Escape character is '^]'. 220 weasel.finninday.net ESMTP Postfix (Ubuntu) ehlo weasel.finninday.net 250-weasel.finninday.net 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-AUTH LOGIN PLAIN DIGEST-MD5 CRAM-MD5 250 8BITMIME quit </pre>

It even offers the correct services to remote machines:

<pre> [rday@snapper ~]$ telnet finninday.net 25 Trying 24.21.185.50... Connected to finninday.net. Escape character is '^]'. 220 weasel.finninday.net ESMTP Postfix (Ubuntu) ehlo weasel.finninday.net 250-weasel.finninday.net 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-AUTH LOGIN PLAIN DIGEST-MD5 CRAM-MD5 250 8BITMIME quit </pre>

I found another recipe that said it was actually tested on Dapper Drake and correctly identified the sasl2 package that I stumbled over before.

https://help.ubuntu.com/community/Postfix

So I followed that recipe and made these changes to my main.cf:

<pre> root@weasel:/etc/postfix# diff main.cf.orig main.cf 40,41c40,41 < #smtpd_sasl_auth_enable = yes < smtpd_sasl2_auth_enable = yes --- > smtpd_sasl_auth_enable = yes > #smtpd_sasl2_auth_enable = yes 55c55 < smtpd_sasl_local_domain = $mydomain --- > smtpd_sasl_local_domain = 59a60,61 > smtp_use_tls = yes > smtp_tls_note_starttls_offer = yes </pre>

And restarted postfix.

When I try to send an email, I get this in the logs:

<pre> Apr 21 15:52:15 localhost postfix/smtpd[26421]: connect from PSMFC-fwgt.psmfc.org[205.230.28.193] Apr 21 15:52:15 localhost postfix/smtpd[26421]: setting up TLS connection from PSMFC-fwgt.psmfc.org[205.230.28.193] Apr 21 15:52:15 localhost postfix/smtpd[26421]: TLS connection established from PSMFC-fwgt.psmfc.org[205.230.28.193]: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits) Apr 21 15:52:23 localhost postfix/smtpd[26421]: warning: SASL authentication failure: no secret in database Apr 21 15:52:23 localhost postfix/smtpd[26421]: warning: PSMFC-fwgt.psmfc.org[205.230.28.193]: SASL CRAM-MD5 authentication failed Apr 21 15:52:23 localhost postfix/smtpd[26421]: warning: SASL authentication failure: cannot connect to saslauthd server: No such file or directory Apr 21 15:52:23 localhost postfix/smtpd[26421]: warning: SASL authentication failure: Password verification failed Apr 21 15:52:23 localhost postfix/smtpd[26421]: warning: PSMFC-fwgt.psmfc.org[205.230.28.193]: SASL PLAIN authentication failed Apr 21 15:52:23 localhost postfix/smtpd[26421]: warning: SASL authentication failure: cannot connect to saslauthd server: No such file or directory Apr 21 15:52:23 localhost postfix/smtpd[26421]: warning: PSMFC-fwgt.psmfc.org[205.230.28.193]: SASL LOGIN authentication failed Apr 21 15:52:46 localhost postfix/smtpd[26421]: disconnect from PSMFC-fwgt.psmfc.org[205.230.28.193] </pre>


Made a few other changes to /etc/default/saslauthd:

<pre> root@weasel:/etc/default# diff saslauthd.orig saslauthd 3a4,7 > PWDIR="/var/spool/postfix/var/run/saslauthd" > PARAMS="-m ${PWDIR}" > PIDFILE="${PWDIR}/saslauthd.pid" > 10,11c14,15 < #PARAMS="-m /var/spool/postfix/var/run/saslauthd -r" < PARAMS="-m /var/run/saslauthd" --- > > OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd" </pre>

That got things working and I could suddenly see that my certificate is expired. But I found that attempts to send TLS to my upstream provider, comcast were failing, so I took out the smtp_enable_tls.

This is the config that I've wanted. Now I can configure our Thunderbird on the laptop to be able to send mail wherever it is on the net:

outgoing server: weasel.finninday.net
port: 25
secure connection: TLS
Use username and password.

Going further, I took out md5 from the ciphers listed in /etc/postfix/sasl/smtpd.conf and commented out 

#allow_plaintext:true

Another postfix reload.

Looks good. Now the logs are pretty clean:

<pre> Apr 21 16:22:02 localhost postfix/smtpd[28590]: connect from PSMFC-fwgt.psmfc.org[205.230.28.193] Apr 21 16:22:02 localhost postfix/smtpd[28590]: setting up TLS connection from PSMFC-fwgt.psmfc.org[205.230.28.193] Apr 21 16:22:04 localhost postfix/smtpd[28590]: TLS connection established from PSMFC-fwgt.psmfc.org[205.230.28.193]: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits) Apr 21 16:22:05 localhost postfix/smtpd[28590]: 18DBC1334439: client=PSMFC-fwgt.psmfc.org[205.230.28.193], sasl_method=PLAIN, sasl_username=xxxx Apr 21 16:22:05 localhost postfix/cleanup[28597]: 18DBC1334439: message-id=<480D2198.6020507@finninday.net> Apr 21 16:22:05 localhost postfix/qmgr[28565]: 18DBC1334439: from=<xxxx@finninday.net>, size=649, nrcpt=1 (queue active) Apr 21 16:22:05 localhost postfix/smtpd[28590]: disconnect from PSMFC-fwgt.psmfc.org[205.230.28.193] Apr 21 16:22:19 localhost postfix/smtpd[28607]: connect from localhost.localdomain[127.0.0.1] Apr 21 16:22:19 localhost postfix/smtpd[28607]: 9676713348D9: client=localhost.localdomain[127.0.0.1] Apr 21 16:22:19 localhost postfix/cleanup[28597]: 9676713348D9: message-id=<480D2198.6020507@finninday.net> Apr 21 16:22:19 localhost postfix/qmgr[28565]: 9676713348D9: from=<xxxx@finninday.net>, size=1112, nrcpt=1 (queue active) Apr 21 16:22:19 localhost postfix/smtpd[28607]: disconnect from localhost.localdomain[127.0.0.1] Apr 21 16:22:19 localhost amavis[19645]: (19645-06) Passed CLEAN, [205.230.28.193] [205.230.28.193] <xxxx@finninday.net> -> <xxxx@psmfc.org>, Message-ID: <480D2198.6020507@finninday.net>, mail_id: o1siW-0+w6ed, Hits: -3.343, 14426 ms Apr 21 16:22:19 localhost postfix/smtp[28598]: 18DBC1334439: to=<xxxx@psmfc.org>, relay=127.0.0.1[127.0.0.1], delay=15, status=sent (250 2.6.0 Ok, id=19645-06, from MTA([127.0.0.1]:10025): 250 Ok: queued as 9676713348D9) Apr 21 16:22:19 localhost postfix/qmgr[28565]: 18DBC1334439: removed Apr 21 16:22:20 localhost postfix/smtp[28608]: Host offered STARTTLS: [smtp.g.comcast.net] Apr 21 16:22:20 localhost postfix/smtp[28608]: 9676713348D9: to=<xxxx@psmfc.org>, relay=smtp.g.comcast.net[76.96.30.117], delay=1, status=sent (250 2.0.0 GPNK1Z00P15fmCg8U00000 mail accepted for delivery) Apr 21 16:22:20 localhost postfix/qmgr[28565]: 9676713348D9: removed </pre>

Troubleshooting

Can't send mail from Dad's house in San Diego

Attempts to send mail from remote locations just hang and eventually time out. There are no entries in the mail logs about failures. The connection must be not even reaching the machine. The firewall is set to allow connections to port 25, but nmap says this: <pre> Interesting ports on c-24-21-185-50.hsd1.mn.comcast.net (24.21.185.50): Not shown: 1706 filtered ports PORT STATE SERVICE 21/tcp open ftp 22/tcp open ssh 80/tcp open http 143/tcp open imap 389/tcp open ldap 443/tcp open https 445/tcp closed microsoft-ds 993/tcp open imaps </pre> So the firewall isn't really allowing port 25 through.

Ahah, the problem was that the ISP that my father-in-law uses is blocking outbound port 25 traffic. Does it actually reduce spam or just annoy people? They also block outbound traffic on all unprivileged ports. My proposed solution is to use one of the ports they don't block, like ftp, and try to redirect smtp from that port to my port 25. SSH has a way to do this: 

ssh -R 21:localhost:25 rday@localhost

But that doesn't quite work since it is a privileged port, so I have to be root, but then there is some config option that is preventing root from redirecting ports. If I can't figure out the ssh config, I'll have to use some other option.

Redirecting to a non-privileged port works just fine: 

ssh -R 2025:localhost:25 rday@localhost

But unfortunately, that is blocked by the ISP.

I may resort to using iptables to do the redirect using something like this: 

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 21 -j REDIRECT --to-port 25

Blocked by my own ISP

For my own protection, Comcast shut off my email. <pre> Subject: Customer Security Assurance Notice

ACTION REQUIRED: Comcast has determined that your computer(s) have been used to send unsolicited email ("spam"), which is generally an indicator of a virus. For your own protection and that of other Comcast customers, we have taken steps to prevent further transmission of spam from your computer(s). </pre>

According to my mail server logs I've sent 100 emails in the previous 24 hours. The email says nothing of the amount of mail that triggers the spam warning, but I've heard other people say it is 1000 in a 24 hour period. All my anti-virus reports are clean. I think the problem was that I've been sending mail to smtp.comcast.net:25 for the past few years. At some point they changed policy and said that mail should go to port 587, but I didn't get the memo.

By the way, I talked to Comcast tech support before I knew what was happening and he told me that the problem was my yahoo mail account (which I don't have) or my home mail server. I asked if it might have something to do with being blacklisted for sending spam and he said no, definitely not that. He asked if I had checked the message boards to answer this problem and I said yes, everyone else with my symptoms has been blocked for sending too many emails. He said Comcast has a policy of not filtering ports. One useful thing (that is, one thing that wasn't completely false) that I got out of him was the details I needed to log into my long-unused Comcast webmail account which was how I found the email that I quoted above.

Now when I try to connect to comcast to send mail, it looks like this: 

rday@weasel:~$ telnet smtp.comcast.net 25
Trying 76.96.30.117...

But if I try the same thing from work, I get this: <pre> [rday@snapper bin]$ telnet smtp.comcast.net 25 Trying 76.96.30.117... Connected to smtp.comcast.net. Escape character is '^]'. ehlo localhost 220 OMTA09.emeryville.ca.mail.comcast.net comcast ESMTP server ready 250-OMTA09.emeryville.ca.mail.comcast.net hello [69.30.63.162], pleased to meet you 250-HELP 250-AUTH LOGIN PLAIN CRAM-MD5 250-SIZE 15728640 250-ENHANCEDSTATUSCODES 250-8BITMIME 250-STARTTLS 250 OK quit 221 2.0.0 OMTA09.emeryville.ca.mail.comcast.net comcast closing connection Connection closed by foreign host. </pre> So I'm on some kind of blacklist for talking to port 25. Current Comcast documentation says that you should use port 587 for sending mail. But I can't find their instructions on how to do that on their website.

I was able to verify that my home machine can connect to smtp.comcast.net:587 if I authenticate via TLS: <pre> rday@weasel:~$ telnet smtp.comcast.net 587 Trying 76.96.62.117... Connected to smtp.g.comcast.net. Escape character is '^]'. 220 OMTA04.westchester.pa.mail.comcast.net comcast ESMTP server ready ehlo localhost 250-OMTA04.westchester.pa.mail.comcast.net hello [24.21.185.50], pleased to meet you 250-HELP 250-AUTH LOGIN PLAIN CRAM-MD5 250-SIZE 15728640 250-ENHANCEDSTATUSCODES 250-8BITMIME 250-STARTTLS 250 OK auth plain {magic-mmencoded-token} 235 2.7.0 ... authentication succeeded quit 221 2.0.0 OMTA04.westchester.pa.mail.comcast.net comcast closing connection Connection closed by foreign host. </pre> To come up with the magic auth token, I used this formula: 

[rday@snapper bin]$ perl -MMIME::Base64 -e \
> 'print encode_base64("\0username\0password");'

Now I need to tell postfix to connect that way for sending mail. I was able to direct postfix to use that port by these directives in main.cf (note the key word "submission" to indicate port 587): 

relayhost = [smtp.comcast.net]:submission
smtp_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_auth_enable = yes

After putting the username and password in sasl_password, I had to make it a db file like so: 

postmap /etc/postfix/sasl_passwd

More details about how that works are here: http://www.postfix.org/SASL_README.html#client_sasl

In order to receive mail, I'll have to probably buy a port deflector service like they describe in this post: http://www.lockergnome.com/usrbingeek/2007/04/12/comcast-blocking-port-25/ Here is a better link to the no-ip reflector service: http://www.no-ip.com/support/guides/email/blocked_port_25.html

I spent a quick $40 on no-ip.com reflector service, repointed the MX records for my domain to no-ip.com, and started up mail service on a non-standard port. And it works! Now Comcast has to go to a lot more trouble to shut off my email. And in the mean time, it looks like it will be feasible to switch to business class DSL with a static IP address and non-filtered ports and everything. Yay. Thanks Comcast, for finally pushing me into better, cheaper service with a different provider.

After a few weeks of using the mail reflector service, I was considering using it even after I switch to a non-filtered ISP because the spam filtering they do made a drastic reduction in the spam filtering I had to do. I was bouncing around 95% of all inbound mail because it didn't follow mail specifications. Now I am bouncing between 0 and 1%. But alas, a real email was bounced by the mail reflector, so I guess I'll have to save my $40 and do my own filtering so I don't have to worry about false positives.

postfix config for relaying outbound mail to gmail

My ISP (TDS) is dropping mail relaying service for my business-class account. And, by they way, they block all outbound port 25 traffic that doesn't go to their mail relay. So I'm screwed^H^H^H^H^H^H^H motivated to come up with a different way to inject my outbound mail into the internets.

My strategy is to instruct postfix to deliver mail to gmail like this: 

main.cf:
relayhost = [smtp.gmail.com]:587

But it is not quite so easy as that, since I have to authenticate to gmail. Hopefully the following steps will take care of all the fiddly details of doing the authentication properly.

My recipe is based on these documents:

First step, does my current ISP block outbound port 587?

I wouldn't put it past them, but this looks like it is not a problem. 

rday@weasel:~$ nc smtp.gmail.com 587
220 mx.google.com ESMTP n9sm13393068wag.23

changes to main.cf

The following suggested lines for main.cf assume that there are multiple local mail servers and that only the upstream server (gmail.com) should be firmly encrypted. I don't have multiple local mail servers, so I'll tweak it a bit. These are the relevant differences between my current main.cf and the proposed main.cf: <pre> smtp_generic_maps = hash:/etc/postfix/generic smtp_sasl_security_options = noanonymous smtp_sasl_tls_security_options = noanonymous smtp_tls_CAfile = /etc/postfix/cacert.pem smtp_tls_cert_file = /etc/postfix/FOO-cert.pem smtp_tls_key_file = /etc/postfix/FOO-key.pem smtp_tls_loglevel = 1 smtp_tls_per_site = hash:/etc/postfix/tls_per_site smtp_tls_session_cache_database = btree:/var/run/smtp_tls_session_cache smtpd_enforce_tls = no smtpd_sasl_application_name = smtpd smtpd_sasl_auth_enable = no smtpd_sasl_local_domain = $myhostname smtpd_tls_ask_ccert = yes smtpd_tls_session_cache_database = btree:/var/run/smtpd_tls_session_cache transport_maps = hash:/etc/postfix/transport </pre>

Specifically, I'll drop smtp_tls_per_site, smtp_generic_maps, and transport_maps. I'll modify smtpd_enforce_tls to yes.

changes to master.cf

I'm assuming that no changes are necessary to master.cf, but I'm not sure about this: 

relay     unix  -       -       n       -       -       smtp
       -o smtp_generic_maps=

I don't have any other mail servers on my network, so I'm not going to make any changes to master.cf. I want all outbound mail to go to gmail and to have the address translation happen appropriately.

certificates

I'm assuming that I can continue to use my existing certificates, with the small addition of the Thawte credentials. 

smtpd_tls_CAfile = /etc/postfix/cacert.pem
smtpd_tls_cert_file = /etc/postfix/newcert.pem
smtpd_tls_key_file = /etc/postfix/newreq.pem

I'll just add the following entries to main.cf: 

smtp_tls_CAfile = /etc/postfix/cacert.pem
smtp_tls_cert_file = /etc/postfix/newcert.pem
smtp_tls_key_file = /etc/postfix/newreq.pem

update sasl_passwd

I added my gmail username and password to the sasl_password file and then did a postmap on it to create the sasl_passwd.db. I verified that the permissions on these two files allow root and postfix to read them and no one else.

troubleshooting

My first try looks like this: <pre> Aug 18 15:39:58 weasel postfix/smtp[22555]: certificate verification failed for smtp.gmail.com[209.85.147.109]:587: untrusted issuer /C=US/O=Equifax/OU=Equifax Secure Certificate Authority Aug 18 15:39:58 weasel postfix/smtp[22555]: warning: SASL authentication failure: No worthy mechs found Aug 18 15:39:58 weasel postfix/smtp[22555]: 8B3E61238F96: to=<rday@psmfc.org>, relay=smtp.gmail.com[209.85.147.109]:587, delay=1, delays=0.01/0.03/1/0, dsn=4.7.0, status=deferred (SASL authentication failed; cannot authenticate to server smtp.gmail.com[209.85.147.109]: no mechanism available) </pre> So what is wrong with my cert?

After a few tweaks to main.cf, I tried again and got this: <pre> Aug 18 15:52:06 weasel postfix/smtp[24505]: setting up TLS connection to smtp.gmail.com[209.85.147.109]:587 Aug 18 15:52:06 weasel postfix/smtp[24505]: certificate verification failed for smtp.gmail.com[209.85.147.109]:587: untrusted issuer /C=US/O=Equifax/OU=Equifax Secure Certificate Authority Aug 18 15:52:06 weasel postfix/smtp[24505]: Untrusted TLS connection established to smtp.gmail.com[209.85.147.109]:587: TLSv1 with cipher RC4-MD5 (128/128 bits) Aug 18 15:52:11 weasel postfix/smtp[24505]: 69CE01238F96: to=<rday@psmfc.org>, relay=smtp.gmail.com[209.85.147.109]:587, delay=6.1, delays=0.01/0.03/1.7/4.3, dsn=2.0.0, status=sent (250 2.0.0 OK 1250635931 m30sm13533773wag.69) Aug 18 15:52:11 weasel postfix/qmgr[23857]: 69CE01238F96: removed </pre>

The mail got sent, but I want to get rid of the "certificate verification failed" error. I think the error is indicating that my server doesn't trust the body that issued google's certification. That means I have to import some cert somewhere to get it to be trusted. But I think gmail saw no problems on their side, so I have a working config with a wart in the logs that I need to clean up.

summary of changes

<pre> rday@weasel:~$ diff prior.txt now.txt 21c21 < relayhost = [smtp.tds.net] --- > relayhost = [smtp.gmail.com]:submission 23a24,29 > smtp_sasl_security_options = noanonymous > smtp_sasl_tls_security_options = noanonymous > smtp_tls_CAfile = /etc/postfix/cacert.pem > smtp_tls_cert_file = /etc/postfix/newcert.pem > smtp_tls_key_file = /etc/postfix/newreq.pem > smtp_tls_loglevel = 1 24a31 > smtp_tls_session_cache_database = btree:/var/run/smtp_tls_session_cache </pre>

From address rewriting

Sending mail through TDS results in a proper header like this: 

From: rday@myhost.net

but sending via gmail results in 

From: mygmailaccount@gmail.com

Bummer. Luckily my postfix configuration makes it easy to switch back and forth between relayhosts by changing a single line in main.cf.

From what I understand of the documentation provided by Google, you can only send mail through gmail servers for an account that has been registered (with server, username, and password) with the sending gmail account. There are quite a few accounts on my home mail server and I don't feel good about putting all those passwords into Google's keeping. This is where I guess I have to admit that it was too good to be true that gmail would allow me to inject my server's mail into their system.

My next option is to set up an SSH tunnel to work. After that, I think I have to go back to no-ip.com and use their mail server.

SSH tunnel to snapper

The idea is to have weasel's mail server send mail to snapper which will then forward it on to the world. But I have to jump around the firewall between the two mail servers. I can ssh between the two.

The idea is that snapper is running postfix and weasel is running postfix. Weasel's config looks like this: 

relayhost = [snapper.psmfc.org]:submission

Submit email to spiritone.com

Ahh, I am back to dealing with a sane ISP. I was issued a static IP address and don't have to do anything weird to route my mail around. I only had to make these changes to get it to work:

main.cf: 

relayhost = [mail.spiritone.com]:submission

sasl_passwd: 

[mail.spiritone.com]:submission user:password

And then issue these commands: 

postmap sasl_passwd
/etc/init.d/postfix restart
Retrieved from "https://finninday.net/wiki/index.php?title=Smtp_auth&oldid=3195"