Actually, it was timing out.

So I hooked up a monitor and a keyboard to this old old P1/133 with 32MB of ram that had been taken out of a dumpster to be our router to see what was going on.

A few months prior to this I had been playing around with rrd and started generating traffic graphs so that I could get pretty pretty pictures of our traffic (and to figure out when one of my roommates was streaming his porn).

I had set the cron jobs to run every 10 minutes. Initially the data sets were so small that this wasn't an issue, but over time the database got larger and suddenly the cron job was taking 10 minutes and 10 seconds, and then 10 minutes and 20 seconds... on and on until finally cron jobs were running over cron jobs.

So after about 2 minutes waiting to get logged in, I finally get the load average for the system and it was somewhere around 146.20. After another couple of minutes, I was able to disable the cron jobs and kill off rrd graphs. In another 10 minutes, the load average was back to < .70.

All this time it was passing traffic without a hiccup and I believe it had another 100+ days of uptime after this before I moved out. What a box. I miss that router.

Now, admittedly, I've never worked on big networks. But, I know what I like, and I know working on a Cisco feels like 1996, and that working on a full-blown modern UNIX/Linux system feels like home. And, fast network gear is available for Linux and the BSDs. 10Gb interfaces that work with Linux/BSD are not difficult to find, and 1Gb interfaces (sometimes several of them) have been standard on every server and barebones rackmount on the market for a decade.

I understand there are special purpose boxes that do things no off-the-shelf commodity box can do (I don't know enough about high end networking to know where to draw that line), but I honestly doubt 95% of business networks need them. And, yet, a huge percentage of business networks that have real routers (rather than little wifi routers from LinkSys/NetGear or whatever) have a Cisco of some sort, even when they don't have anyone on staff who knows how to work the thing. I've walked into several businesses that had expertise in house for everything except their router, which is also weird...why pick a Cisco router for your little ~100-1000 node network, if you've got Linux or BSD guys hanging out in the server room every day? They'd call someone and sit around waiting when their network was acting up.

I learned how to use Cisco routers just so I'd be able to replace them easily (enough knowledge to read the routing tables, firewall rules, etc., but not enough to do much useful), when I found situations like that.

Or with a generous budget, you can do what BT have done with their 21CN project: For each macro-block (access, aggregation, core), have two redundant networks each one built by a different set of vendors.

Why on earth were you running a hosting business with Cisco 2950s? Those are wiring closet desktop switches with tiny buffers. That alone probably caused a lot of pain. You didn't know about it, but they did.

Second, the Juniper J4350 is a software router based on BSD. There's no special hardware in there like ASICs. It's just a PC.

JunOS (historically) is based on BSD. They're moving to Linux now.

I love OpenBSD for small projects too, but let's admit that there was some bad logic in this decision making process. This could be called "how the mx10 saved my ass because I went with undocumented, unproven open source project that has been on github for 3 days" if the technologies were reversed.

The only time it failed -- the ONLY time -- was due to a full hard disk. Why was the hard disk full? Silly me, I forgot to implement some sort of logging discipline and a couple years' worth of logs of attempted exploits (mainly Windows NT exploits) glancing harmlessly off OpenBSD's hull had filled the disk.

Doing Enterprise IT Arch work, I turned down more than one hot new system and put others through significant lab work, to avoid this fun. It's hard, it's expensive and it's why I wish more hardware was open.

Honestly I wonder, who the hell buys these overpriced networking appliances? Similarly with expensive proprietary databases, or ESB systems. Are they just leftovers of a different time (note article is from 2007)? Is it a case of management buying from the salesman who bought them a nice dinner? Some place where these things generate actual value that I'm missing?

Do you have details on this? Given how active they are in the FreeBSD community, I would have thought to have heard about that already.

Later on, I was working a network overhaul at a private college dedicated to video game creation (DigiPen). Each student was provided a desktop with their own roaming profile. These profiles got huge because there were things like 3D Studio Max projects being transferred to whatever computer the student happened to be at during that period. So we're talking about hundreds of students signing on at about the same time & massive amounts of data being transferred in a very short time. The old network topology was melting down on a regular basis causing the entire campus to come to a halt.

We fixed that by putting in a pair of -- you guessed it -- OpenBSD/PF routers. Now this wasn't some "oh crap" kind of bandaid fix; we had planned this all out & implemented over the summer months when there were no students. It was all planned out (the only problem was that the original plan called for FreeBSD & PF, which crashed -- once we had OpenBSD running, it was 'set it & forget it.')

Maybe I'm just too dumb to know the difference, but those 2950s and OpenBSD routers helped me sleep a lot better at night... they were perfect for what they were intended to do.

And as more than a decade of experience since has proven, the right tool for the job is often the one that gets scoffed at as a "toy" or good "for small projects."

Do you have a source?

Appliances promise all sort of fancy features that might be hard to exactly replicate yourself (if you actually need them often is a different question, but often networking isn't exactly in a position to say "no" to such requests). Although there is more and more a trend of manufacturers also offering the software for virtual environments, many appliances are x86-servers anyways.

+ as the article mentions all those cases where you need specialised hardware, e.g. routing/switching at high speeds, or working in special environments.

people that need to switch more packets than what you buy off of newegg can handle.

edit: oh that and when you have a network of thousands of routers and switches that you want to centrally manage in a way that "mass ssh" doesn't scale to and you just want to buy the management software from someone that has written it before and not spend a while getting the software wrong before you get it right.

if you can count the number of routers you need in your head, you probably should just be using beige boxes with a commodity OS...

I've seen this many times and it will continue to happen.

What's latency like on commodity hardware vs dedicated equipment? I've no experience but would expect the dedicated hardware to be better. BSD is a monolithic kernel, so you'd have the overhead of passing through the for each of the routed packets, right?