Monday, December 11, 2006

Platform specific IP routing behavior


The code I'm working on manipulates routing tables on three different platforms: Linux, Windows and Solaris. Each of them has a different behavior for different scenarios. Here I attempt to document those differences.
First some definitions:
  • Interface A device which can directly reach a subnet via ARP or other protocols. An example is eth0.
  • Direct route A route which indicates which Interface to use to reach a directly connected subnet.
  • Gateway route A route which indicates a gateway to use to reach a subnet which is connected via a router.
  • Default route A special case of a Gateway route in which the destination subnet is all possible addresses.
  • Host route A special case of any of either a Direct route or a Gateway route in which the destination is a single machine.
  • Multicast route A special case of a Direct route in which the destination subnet is the multicast address space, 224.0.0.0/8 or a subset thereof.
Note that example route entries in this table is based on the format emitted from Linux's /sbin/ip/route.
Linux 2.6Windows 2003Solaris 10
The interface chosen to access a gateway via a route is determined by traversing the route table, not hardcoded into the route entry.
For instance, default via 192.168.0.1 does not need the specification dev eth0, because that is determined by finding the direct route 192.168.0.0/24 dev eth0.
nonoyes1
A Gateway route which is also a Host route can be added where the destination is an address that exists in a subnet of another Direct route.
For instance, the route 192.168.0.20/32 via 192.168.0.1 dev eth0 exists while the direct route192.168.0.0/24 dev eth0 also exists.
yes2yes2yes2
A Direct route which is also a Host route can be added where the destination is an address that exists in a subnet of another Direct route.
For instance, the route 192.168.0.20/32 dev eth0 exists while the direct route 192.168.0.0/24 dev eth0 also exists.
yesyesyes
Direct routes can be deleted.yesyes3yes
Route priority can be programmatically controlled.yesyesno4
When an interface is administratively taken down, do the associated Direct route entries disappear?yesyesyes
When an interface is administratively taken down, and associated Direct route entries disappear, do they return when the interface is brought up again?yesyesyes
When an interface is administratively taken down, do the associated Gateway route entries disappear?yesyesyes
When an interface is administratively taken down, and associated Gateway route entries disappear, do they return when the interface is brought up again?noyesno
When an interface is administratively down, is it an error to add a route that references that interface?yesyesyes
When an interface is unplugged, do the associated route entries disappear?noyesno
When an interface is unplugged, and associated route entries disappear, do they return when the interface is plugged in again?N/AyesN/A
When an interface is unplugged, is it an error to add a route that references that interface?noyes5no
When an interface is unplugged, and associated route entries do not disappear, will an alternate route be chosen because the interface is unplugged?noN/Ano
If two interfaces are connected to the same subnet, will ARPrespond on either interface for an address on one of the interfaces?yesnono
Can routes be modified for all attributes including priority? An answer of no means they must be destroyed and recreated to modify attributes.noyesno
Does the operating system create Multicast routes by default?noyesno
Can multicast routes be deleted?yesyes3yes
If Multicast routes do not exist, do multicast packets exit the machine? To where?yes6nono
Can two routes be created with the same destination and priority, but a different interface?noyesyes7
Can a Gateway route be specified with an interface, where that interface does not have a Direct route for the gateway in the Gateway route?no8yes9no8
When you remove a Direct route which is required by a Gateway route to reach the gateway, does the Gateway route disappear?nonono
If you specify a Gateway route with a gateway that is not reachable via a Direct route, is this allowed?noyes10no
1 This configuration is possible if the route entry is set for this behavior. The route entry can also be configured for a specific interface.
2 When a ping is performed on the destination address, it is sent to the gateway, not via the direct route; this is what should be expected by following the rules in reading a route table. However, the gateway in my test was a Linux 2.6 machine, and it rejected the ping with an ICMP of "unreachable." This means such a configuration is possible, but useless.
3 One cannot directly delete a default route or other "protected" routes, but there is a way to fool Windows into deleting it. I found this fascinating discussion
4 The metric attribute cannot be set for a route in Solaris.
5 The error that's returned is ERROR_INVALID_PARAMETER. That doesn't differentiate this condition from other problems.
6 It appears to choose the first available interface.
7 This question is partly irrelevant in Solaris; the priority or metric cannot be set for a route. However, you can create two routes with the same destination but different interfaces.
8 It works even if the Direct route is on another interface, but it must exist.
9 You can set the route, but it doesn't do anything.
10 This strange behavior is apparently allowed; the source address that's used is the address on the interface that is preferred for the Default gateway.