10g Upgrade

11 points by klaxzygen 88 days ago | 15 comments

Almondioco 85 days ago |

I find this sentences very weird:

"the hardware was basically dying, choking on the all the heavy backup traffic that wasn't built to handle gracefully"

Its cleary hardware degeneration nothing to be philosophical about. Its a silicon which routes traffic. It shouldn't 'choke on any type of traffic and it shouldn't die because of it.

Otherwise i'm still not switching to 10g due to energy consumption. Its the switches which need more (if you use copper) and the client ports too.

But i only have one big machine which I do backups to and not even regularly enough that it matters too much.

Fancy hardware isn't more fun if you don't use the features. Spanning tree didn't work properly in my setup. At the end I only use vlan and the whole chain needs to support it. And the only reason for that is dataprotection: Not trusting random shitty IoT devices and i do not want to give them access to my local network.

I do like ubiquity. The switch and Acess Points have great price point and work very well and in comparision to other manufacturres an actual usable good UI.

Havoc 85 days ago |

You don't need two fans to keep a 10gig switch cool. Trick is to be careful with which RJ45 to SFP+ transceivers you buy - they come in hot and not so hot flavour. The ones I've got run at ~60C without any cooling whatsoever.

Tricky bit is the labelling on these tend to be rather rubbish esp on aliexpress so hard to tell what the chipset inside is or what the wattage is.

sekh60 85 days ago | |

Wiitek is a brand with the lower wattage ones that run around 60°C, not sure if they are on AliExpress, but they are on amazon.ca and worked perfectly in my Mikrotik. To OP and other readers, if getting into switches with transceivers be aware that many enterprise networking gear vendors vendor lock their ports and transceivers. That was an expensive lesson for me. There are third parties selling off-brand transceivers for a lot cheaper than the branded ones (fs.com is probably the biggest and best known). This can get complicated if you have different manufacturers and need different coded transceivers or even DACs with different "brands" on each end.

Havoc 85 days ago | | |

Yeah those ones. - Broadcom chipset I believe. I did end up finding much cheaper non Wittek branded ones too on aliexpress but bought a couple different so can’t confidently share link. Think the key was looking for the ones marked Broadcom and 80m?

> vendor lock

That’s one advantage of aliexpress- it’s all unlocked

bombcar 85 days ago | | |

It’s an entire new world of weird things; I have some “ancient” Nortel PoE gigabit switches with 10G uplinks that I was able to find who the actual OEM was, and then find a firmware that could be massaged on.

alexfoo 85 days ago | |

For my own 10G homelab network I jumped the gun and got a couple of Intel X540-T1 cards for my two servers and balked at the cost of the RJ45-SFP+ transceivers (Unifi's version is ~USD60). (I'm sure there are cheaper options for the "not hot" flavour transceivers but I didn't want to have to gamble again.)

In the end I just replaced each X540-T1 with a X520-DA2 which are pretty much the same price on eBay (under USD20) and then I can just use a DAC that's a fraction of the cost of the RJ45-SFP+ transceivers.

Havoc 85 days ago | | |

Yeah its better to avoid rj45 entire for 10gig if one can. It's unavoidable though for some places. e.g. A lot of consumer boards have a spare X1 PCIE slot...which is enough for 10gig if its gen4.

...but all the 1X cards I've seen are rj45 and didn't want to take a chance on seeing if a X4 sfp card can run in a 1X at full 10gig

sekh60 85 days ago |

Nice score on the used switch! I used a 10GbE Mikrotik up until recently until it started having hardware failures (my second one, a few years older is running perfectly, been very happy with it). I got exceptionally lucky finding a used 64 port 40/100Gbps enterprise switch on ebay for $1000CAD (same switch here normally goes for $3-4k CAD used). Most of my ceph cluster is limited to 10Gbps (onboard RJ45 for 4/5 nodes, and the PCI-E slot on each is taken by NVMe drives), but I feel I'm already outgrowing that speed, my few nodes already saturate 10Gbps - NVMe drives are fast! Now I just gotta wait for 100GbE NICs to come down in price, I'm not sure I'm ready to buy a used NIC yet (was nervous about the switch, first used piece of network gear for me), but the knock off Intels are like $350 minimum here for a 2 port.

bombcar 85 days ago | |

The only “new” NIC in my setup is the one in the 10G thunderbolt dock for my Mac, everything else was used and has worked perfectly.

sekh60 85 days ago | | |

Good to know! Not sure why I am so often shy on getting used. I've done it when the deals are simply very good, but that's just been with referb harddrives (with a warranty) and this switch, which was just too good of a deal. I gotta get braver there.

fdomingues 85 days ago |

What are the use cases for multi gbps home network or internet access? With a 1/1 Gbps link we could use 500 Mbps to transfer 5 TBytes per day of home assistant backups, time machine deltas, security footage..., and still have the other 500 Mbps to do whatever we want.

zokier 85 days ago |

fyi zyxel has some reasonably priced 2.5G/10G base-t switches with or without poe++. I got myself XS1930-12HP when I updated my home network to wifi6 and 10G. The biggest annoyance was that some features are gated behind "advanced" license, but iirc nothing too critical. Small things, but I like that it has integrated power supply so it just has standard iec connector in the back instead of needing a power brick. I did replace the standard 40mm fans with 120mm noctua; the airflow might not be optimal but I think it'll survive.