Two isp routing

I have two ISPs. I didn't plan on getting two ISPs. I've had one for ages that I'm reasonably happy with that I've shopped long and hard for. It gives me a static IP and doesn't charge too much. It has a 1Mbps down and 370Kbps up.

Then my son wanted to vastly upgrade our connectivity and our existing link just couldn't fill the bill. So we got another. His link is 30Mbps down and 3Mbps up. I want some of that. But I don't want to disrupt the existing services on my static IP and the original link.

My routing table looks like this now.

root@weasel:/# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         216.99.216.1    0.0.0.0         UG    0      0        0 eth0
10.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eth1
216.99.216.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0

So I'm going to add another gateway (faster link) and make it the default, but leave the existing gateway. Do I need to give it a higher metric? It looks like a lot of my questions will be answered here: http://www.linuxhorizon.ro/iproute2.html

Starting with the default here:

root@weasel:/# cat /etc/iproute2/rt_tables 
#
# reserved values
#
255	local
254	main
253	default
0	unspec
#
# local
#
#1	inr.ruhep

No response from comcast dhcp

The service guy set up the connection to a windows machine that works just fine. When I try to connect with my laptop or server, I get no response from the dhcp server. Actually, that's not quite true. I get an IPv6 address, but not an IPv4. The windows machine gets an IPv4 just fine.

Maybe I have to spoof my mac address to proceed. Here is my old laptop mac address, so I can change it back:

f0:1f:af:16:f3:39

Here is the mac address on the windows machine:

bc:5f:f4:54:c9:d1

Switch laptop mac address to match what comcast expects.

ifconfig em1 hw ether bc:5f:f4:54:c9:d1

ifconfig em1 reports that it was successfully changed.

yay, it worked.

And better yet, it was easy to clean up.

ifdown em1
ifconfig em1

reports that I've got my old mac address again.

Now test on server.

# original mac address here

eth2      Link encap:Ethernet  HWaddr 68:05:ca:19:af:e0  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:19 Memory:fd9c0000-fd9e0000 

root@weasel:~# ifconfig eth2 hw ether bc:5f:f4:54:c9:d1
root@weasel:~# ifconfig eth2
eth2      Link encap:Ethernet  HWaddr bc:5f:f4:54:c9:d1  
          inet6 addr: fe80::6a05:caff:fe19:afe0/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:11837 errors:0 dropped:0 overruns:0 frame:0
          TX packets:870 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:863363 (863.3 KB)  TX bytes:565985 (565.9 KB)
          Interrupt:19 Memory:fd9c0000-fd9e0000 
# plug cable and watch dhcp log

Sep 28 15:36:00 weasel dhclient: DHCPDISCOVER on eth2 to 255.255.255.255 port 67 interval 3 (xid=0x2f2cf17d)
Sep 28 15:36:00 weasel dhclient: DHCPREQUEST of 24.20.234.228 on eth2 to 255.255.255.255 port 67 (xid=0x2f2cf17d)
Sep 28 15:36:00 weasel dhclient: DHCPOFFER of 24.20.234.228 from 73.94.124.1
Sep 28 15:36:00 weasel dhclient: DHCPACK of 24.20.234.228 from 73.94.124.1
Sep 28 15:36:00 weasel dhclient: bound to 24.20.234.228 -- renewal in 1167 seconds.

Awesome.

Make it permanent by defining it in /etc/network/interfaces like this:

auto eth2
iface eth2 inet dhcp
hwaddress bc:5f:f4:54:c9:d1

iptables

I've already made changes to my iptables config to make eth2 and eth0 (the two ISPs) be treated generally the same. But the rules are too restrictive for eth2.

root@weasel:/etc/default# ping -I eth2 google.com
PING google.com (74.125.239.104) from 24.20.234.228 eth2: 56(84) bytes of data.
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted

# open up iptables for eth2

root@weasel:/etc/default# ping -I eth2 google.com
PING google.com (173.194.33.9) from 24.20.234.228 eth2: 56(84) bytes of data.
64 bytes from sea09s01-in-f9.1e100.net (173.194.33.9): icmp_req=1 ttl=56 time=12.0 ms
64 bytes from sea09s01-in-f9.1e100.net (173.194.33.9): icmp_req=2 ttl=56 time=11.0 ms

So I have to think harder about what iptables rules I need. Let's look at the basics. If I leave out the rules for security and network hygiene, it looks like this:

# eth2 = untrusted link to comcast
[0:0] -A POSTROUTING -o eth2 -j MASQUERADE
[0:0] -A FORWARD -m state --state NEW -o eth2 -j ACCEPT
[0:0] -A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT