Upgrade to Maverick

• Started dist upgrade at 5:45pm

I didn't feel like burning a CD this time and I had performed a few successful upgrades relying on the servers. My work desktop managed an upgrade in about an hour and a half.

• traceroute is no longer supported by Canonical?

I got errors during the upgrade about these items

• squid (claimed that it didn't upgrade)
• phpmyadmin
• samba
• something about the updater

But, YAY, media wiki didn't get messed up.

php5 mhash.so

• a vestigial php5 config file was left behind for mhash which caused this error to be emailed to root every 30 min:

PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib/php5/20090626/mhash.so' - /usr/lib/php5/20090626/mhash.so: cannot open shared object file: No such file or directory in Unknown on line 0

The bug is documented here: https://bugs.launchpad.net/ubuntu/+source/php5/+bug/541439/+activity

The fix was to remove the file /etc/php5/cli/conf.d/mhash.ini

two canonical landscape banners

• when I ssh into my machine, I now get two landscape banners, one for lucid and one for maverick:

diana:~ rday$ ssh finninday.net
Linux weasel 2.6.35-22-generic #34-Ubuntu SMP Sun Oct 10 09:26:05 UTC 2010 x86_64 GNU/Linux
Ubuntu 10.10

Welcome to Ubuntu!
 * Documentation:  https://help.ubuntu.com/

  System information as of Tue Oct 19 10:11:06 PDT 2010

  System load:  0.15                Processes:           269
  Usage of /:   15.8% of 223.21GB   Users logged in:     1
  Memory usage: 24%                 IP address for eth0: xxx.xxx.xx.xx
  Swap usage:   0%                  IP address for eth1: 10.0.0.3
  Temperature:  22 C

  Graph this data and manage this system at https://landscape.canonical.com/

2 packages can be updated.
0 updates are security updates.

Ubuntu 10.04.1 LTS

Welcome to Ubuntu!
 * Documentation:  https://help.ubuntu.com/

  System information as of Mon Oct 18 13:54:58 PDT 2010

  System load:  0.06                Processes:           275
  Usage of /:   15.1% of 223.21GB   Users logged in:     1
  Memory usage: 24%                 IP address for eth1: 10.0.0.3
  Swap usage:   6%                  IP address for eth0: xxx.xx.xxx.xx
  Temperature:  22 C

  Graph this data and manage this system at https://landscape.canonical.com/

5 packages can be updated.
0 updates are security updates.

*** System restart required ***
Last login: Tue Oct 19 10:04:09 2010 from somewhere.somewhere.com

This bug is tracked here: https://bugs.launchpad.net/ubuntu/+source/pam/+bug/659738

My fix was to remove the file /etc/motd.tail

Yay! On Nov 8, 2010, my system got the update to package sysvinit-utils to fix this bug.

sysvinit (2.87dsf-4ubuntu19) maverick-proposed; urgency=low

  * debian/initscripts.postinst (LP: #659738):
    - do not create motd.tail from motd on systems using update-motd.
    - clean up incorrectly created motd.tail from upgrade to Maverick.

 -- Kees Cook <kees@ubuntu.com>  Thu, 28 Oct 2010 20:30:54 -0700

amavisd not running

• hobbit says that amavisd is no longer running

synaptic said that amavisd-new was installed, but amavisd-new-postfix was not so I installed amavisd-new-postfix and got this error:

Stopping amavisd: (not running).
Starting amavisd:   The value of variable $myhostname is "weasel", but should have been
  a fully qualified domain name; perhaps uname(3) did not provide such.
  You must explicitly assign a FQDN of this host to variable $myhostname
  in /etc/amavis/conf.d/05-node_id, or fix what uname(3) provides as a host's 
  network name!
(failed).
invoke-rc.d: initscript amavis, action "restart" failed.

I get the same error when manually trying to start amavis:

root@weasel:~# /etc/init.d/amavis start
Starting amavisd:   The value of variable $myhostname is "weasel", but should have been
  a fully qualified domain name; perhaps uname(3) did not provide such.
  You must explicitly assign a FQDN of this host to variable $myhostname
  in /etc/amavis/conf.d/05-node_id, or fix what uname(3) provides as a host's 
  network name! 
(failed).

I love lucid and helpful error statements. I entered a FQDN in the config file and restarted amavis.

There were 29 messages in the queue when I got amavis restarted. Within a minute, the queue started to drain by itself.

Now hobbit reports all services are green.

Hmmm, the mail queue is not draining as fast as I would expect. I'll have to follow the path through my configs again. I wish I had written down how it all works a little bit better.

I removed the amavisd-new-postfix package and restarted, but still no joy. New mail is piling up in the queue without being delivered.

I tried removing amavisd package and restarting postfix. Still mailq reports a pileup and no movement.

My postfix/main.cf still had a reference to amavis content filtering, so I commented that out and restarted postfix. Still a backlog.

Found that only one of two clamav daemons were running, so I started the clamd.

root@weasel:/etc/postfix# ps -eaf | grep clam
clamav   23042     1  0 22:23 ?        00:00:11 /usr/bin/freshclam -d --quiet
clamav   28699     1  0 23:16 ?        00:00:00 /usr/sbin/clamd

Now I see this error in mail.log

Oct 19 23:18:53 weasel amavis[28041]: (28041-02) (!)run_av (ClamAV-clamd) FAILED - unexpected , output="/var/lib/amavis/tmp/amavis-20101019T231354-28041/parts: lstat() failed: Permission denied. ERROR\n"
Oct 19 23:18:53 weasel amavis[28041]: (28041-02) (!)ClamAV-clamd av-scanner FAILED: CODE(0x3039dc0) unexpected , output="/var/lib/amavis/tmp/amavis-20101019T231354-28041/parts: lstat() failed: Permission denied. ERROR\n" at (eval 115) line 594.
Oct 19 23:18:53 weasel amavis[28041]: (28041-02) (!!)WARN: all primary virus scanners failed, considering backups

I think the fix for that was listed in this howto: https://help.ubuntu.com/community/PostfixAmavisNew

In which it states:

If you see the following error in /var/log/syslog when amavisd is trying to scan a message:

amavis[30807]: (30807-01) (!!) ask_av (ClamAV-clamd) FAILED - unexpected result: /var/lib/amavis/tmp/amavis-20070615T125025-30807/parts: lstat() failed. ERROR\n

Try changing the permissions on /var/lib/amavis/tmp:

  chmod -R 775 /var/lib/amavis/tmp

Now mail is flowing again. Oh, I had to give it a kick with the

 mailq -q 

command to tell it to retry everything stuck in the queue.

However, I'm not sure everything is working as desired. I can't see postgrey showing up in the log anymore, but there is a lot of spam noise, so maybe that isn't a problem. I saw a valid mail get marked as spam and I'm not sure how to check the spam box. I really hate false positives.

After changing the permissions on /var/lib/amavis/tmp, I got this error again:

Oct 20 10:20:59 weasel amavis[2985]: (02985-01) (!)run_av (ClamAV-clamd) FAILED - unexpected , output="/var/lib/amavis/tmp/amavis-20101020T102059-02985/parts: lstat() failed: Permission denied. ERROR\n"
Oct 20 10:20:59 weasel amavis[2985]: (02985-01) (!)ClamAV-clamd av-scanner FAILED: CODE(0x311c538) unexpected , output="/var/lib/amavis/tmp/amavis-20101020T102059-02985/parts: lstat() failed: Permission denied. ERROR\n" at (eval 115) line 594.
Oct 20 10:20:59 weasel amavis[2985]: (02985-01) (!!)WARN: all primary virus scanners failed, considering backups

So I check the directory and it contained this:

root@weasel:/var/lib/amavis/tmp# ls -al
total 16
drwxrwxr-x 4 amavis amavis 4096 2010-10-20 10:21 .
drwxr-xr-x 7 amavis amavis 4096 2010-10-20 09:01 ..
drwxrwxr-x 3 amavis amavis 4096 2010-10-20 09:07 amavis-20101020T090732-31087
drwxr-x--- 3 amavis amavis 4096 2010-10-20 10:20 amavis-20101020T102059-02985

I was able to reliably trigger the error by sending an email to my prin email forwarding service. That would cause a transaction between prin's mail server and mine that would cause the amavis/clam tmp permission error.

I realized that I had added amavis to the clamav group and added clamav to the amavis group, but had not restarted those two services. When I restarted clamav and amavis and sent an email through prin, it came just fine. No more errors.

sshd refuses connection from localhost

This pisses off hobbit which tests sshd by trying to connect to localhost. So now ssh is showing as yellow in my hobbit report. I'll get to that later.

I can't blame this on the upgrade. I had localhost in my /etc/hosts.deny. Taking it out made hobbit happy again.

squirrelmail no longer available

It appears that my custom squirrelmail URL was overwritten by the default. I had to make this change to re-enable squirrelmail as my clients expect it:

root@weasel:/etc/apache2/conf.d# head squirrelmail.conf 
#Alias /squirrelmail /usr/share/squirrelmail
Alias /webmail /usr/share/squirrelmail