IP Routing question

[Ry4an Brase <ry4an@acm.cs.umn.edu>]

I spent many hours last night pulling my hair out over this one and
hopefully someone can help:

I've got a box (cerberus) with two NICs.  Each NIC has a cross-over cable
that connects it to antoher nic in a differnt box (bigdaddy and dan).  The
cerberus/bigdaddy link is 100Mb and has LinkSys PCI cards at each end of
the crossover.  The cerberus/dan link is 10Mb and has D Link22x (NE2000
clone) ISA cards at each end of the crossover.

I've got the 100Mb net set up with IPs in the 192.168.100.x range, and the
10Mb set up with IPs in the (you guessed it) 192.168.10.x range.  I've got
the two end nodes (bigdaddy & dan) set up with nothing in their routing
tables except an entry that points the default route towards cerberus. 

cerberus (the one w/ the two NICs) has a routing table like this:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref  Iface
192.168.100.1   *               255.255.255.255 UH    0      0    eth1
192.168.10.1    *               255.255.255.255 UH    0      0    eth0
192.168.100.0   *               255.255.255.0   U     0      0    eth1
192.168.10.0    *               255.255.255.0   U     0      0    eth0
127.0.0.0       *               255.0.0.0       U     0      0    lo

So nearly as I can tell that should be all that's needed for things to
route to each other.

Just for completeness here's the ipchains config on cerberus

Chain input (policy ACCEPT):
Chain forward (policy ACCEPT):
Chain output (policy ACCEPT):

Here are the ifconfig entries for the two NICs in cerberus:

eth0      Link encap:Ethernet  HWaddr 00:80:C8:FA:72:9C  
          inet addr:192.168.10.1  Bcast:192.168.10.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9056 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          Interrupt:3 Base address:0x280 

eth1      Link encap:Ethernet  HWaddr 00:A0:CC:28:45:A2  
          inet addr:192.168.100.1 Bcast:192.168.100.255 Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6426 errors:0 dropped:0 overruns:0 frame:0
          TX packets:73 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          Interrupt:11 Base address:0xe800 

There, with all that as background, let me tell you what is or isn't
working. 

WORKING:
	pinging cerberus to bigdaddy
	pinging cerberus to dan
	pinging dan to cerberus
	pinging bigdaddy to cerberus
	pinging bigdaddy to the 'far' NIC in cerberus
		(ex: 192.168.100.3 (bigdaddy) to 192.168.10.1)
        pinging dan to the 'far' NIC in cerberus
                (ex: 192.168.10.2 (dan) to 192.168.100.1)

NOTWORKING:
	pinging bigdaddy to dan
	pinging dan to bigdaddy


NOTES:

I'm pretty sure the default routes on the end nodes (dan and bigdaddy) are
ok, and the boxen do route all requests through their NICs (flicker in the
activity lights).

GUESSES:

Perhaps I need something to tell cerberus it's ok to connect the two?
Soemthing in the forward rule? A 1 written to a file in /proc? A blood
sacrifice?

Cerberus is running whatever kernel comes with rh 6.1.  Ifn' I get this
working I'll build up a kernel that's optimized for routing, but I didn't
think it was necessary to get it to work at all.

ADVICE I'VE RECEIVED:

"Only high end NICs will hand off packets for routing.  Put Intel, 3COM,
or (real) NE2000 NICs in cerberus."  If that's true why the hell wouldn't
O'Reilly's _TCP Network Administration_ mention that little catch? If I
can't hear any better suggestions I'll get new NICs, but if (and I may be
wrong) routing is done at the OS level what does it matter to the NIC if
the packet will be given to antoher NIC or used at the host?

Thanks in advance for any clock cycles you can waste on this,
Ry4an

--
Ry4an Brase - http://ry4an.org - 612-623-9946
'If you're not a rebel when you're 20 you've got no heart; if
 you're  not establishment when you're 30 you've got no brain.'