Hello,

Here is something that I must ask for guidance from the more experienced
administrators. I must say up front that I am not a networking
specialist. My forte is in our "custom" system.

We have a situation at work where someone misconfigured a switch and it
caused some major failures that were hard to trace. We have a unique
environment where we route custom packets through a brand-new Cisco 4500. We
have new Cisco 3560 switches that distribute links to each of two rooms that
have "custom" digital equipment. This is a new setup for us, and is
mission-critical. We have moved from an analog network to this new
digital UDP based system. In total we have about 14 3560s, and plugged into
these are about 200 other "custom" switches that are vendor-specific.

The point of contention is that someone thought they were doing the
right thing and jumped in where they were not asked to by installing a
new 3560.

  *) This person set the switch up "hot" (on the network)
  *) They used two uplink ports, intending on ganging them togther
  *) They did not properly set the ports into a channel-group

This made the 3560 seem like a router and flooded all of our custom
switches with so much traffic that the devices could not effectively
talk to each other. This would be sorta OK in a TCP environment, but we
have a UDP based system that relies heavily on very low latency.

So here is my question:

I am being pushed by the higher-ups to come up with a software solution
for this problem, which I feel is a process problem. The process should
be to NOT SET THE SWITCH UP ON THE F**KING NETWORK! And to have another
person verify the setup prior to bringing up a new piece of equipment on
the network that is mission-critical. Beyond that the person just went
and did it without coordinating with anyone.

Should I bow to the pressure and force our vendor to "fix" their
software to be able to function in an abnormal network setup? This would
allow certain folks to save face while straining our relationship with
out vendor.

Or

Should I instill a process such that this would never happen again and
put the lock-down on people who configure devices in/on this network?
This involves disallowing the people who are supposed to be the
networking specialists from configuring the "custom" network.

Or

Is there a Cisco configuration that can be used to disallow "unknown"
routers on the VLAN? This seems unlikely to me.


It's one or the other at this point, as we have lost a lot of
credibility in this situation, and we must move forward with
implementation. This is the second time now that a misconfigured switch
has been setup hot on the "custom" network.

Has anyone had a similar situation?

Thanks in advance for your sage advice.


p.s. No, at this point I cannot divulge what the "custom" is.

-- 
As fast as it ever got, it never got fast enough for me.
   Hunter S. Thompson