HomeLab #1: MikroTik as a Home Router(justsomebody.dev) |
HomeLab #1: MikroTik as a Home Router(justsomebody.dev) |
Is there any manufacturer out there who produces high quality hardware that is not locked down?
But if you still want it, MikroTik doesn't try too hard to prevent you from running modified software. You can port OpenWrt, and most modern MikroTik devices use modern enough Linux to use a device tree blob. Try pulling out a dtb from your firmware and ask an LLM to do the initial porting - just don't forget to backup your RouterOS license key first, they are device specific.
https://shop.opnsense.com/product-categorie/hardware-applian...
chokes on coffee
From my old-man, everything-is-stupidly-expensive point of view, this is my scale:
cheap side | reasonable | expensive side | Deciso boxes
For homelab, you are far better served buying a second hand Sophos baremetal/server rack mounted, and installing OPNSense in it.
I have a 2nd hand Sophos SG210 Rev3 from eBay, replaced the Celeron CPU with an i7 one, 16GB, new SSD, dual 10G NIC, it had been running like a dream.
There is no contract with MikroTik. You buy the hardware, that's it. The software and updates are free.
I haven't been able to find many other devices that tick so many boxes. These seem to exist on a class of their own.
Having said that, mikrotik hardware running OpenWRT feels like something that might scratch one of my itches.
The idea is to run OpenWRT passing through the ISP NIC and one (or both) of the SFP+ ones going to the switch, and also run other services in isolated containers.
I'm even thinking about putting Home Assistant on a VM on it but I'm not sure yet what impact it will have on routing performance so that's on hold for now.
It's so much more powerful than what I need for the router/firewall that it should be comfortable to run quite a few things on it, and I can start moving them off it to other devices if Internet speeds catch up to it in the future.
Also, shipping winbox on all platforms has been a huge QoL improvement
Don't get me wrong, they make powerful and feature-complete stuff, but their UI sort of assumes you really know what you're doing. I don't know what I'm doing.
But my LLM does. I've been able to set up site-to-site VPNs with Mikrotik in fifteen minutes now, rather than the day-long exercise of googling and praying.
https://manual.mikrotik.com/llms-full.txt
Their front page documentation specifically mentions it: https://manual.mikrotik.com/docs/introduction#machine-readab...
Is that not just markdown? is there a specific way of formatting docs that's "friendly" to LLMs now?
Nevertheless let's give MikroTik, where it's due:
* new documentation page is much cleaner to browse
* tutorials on their YouTube channel are quite informative
* WinBox v4 has in my opinion much more pleasent UX/UI than v3
That's also my problem. My colleague tells me that it's very easy to setup, but he's a network engineer and knows what things are.
Mikrotik seems to operate in an interesting area between high-end home-labing and the SME market. Without preexisting networking knowledge at a certain level, their documentation isn't all that helpful. I'm sure you can learn from the documentation, YouTube videos and trial and error, but if like me you get a few hours once in a while to take down your home network and get it back up it's going to take a while to gain the required knowledge. I really want an addition internet connection so I can play with this stuff.
On the negative, I'd say it's pretty clear that Mikrotik want you do buy training. It feels like some information is withheld by those with the certifications and Mikrotik and the UI isn't helping you much to figure out stuff on your own.
in worst case you just install the old rule back if it fuck up, no really damage can happen.
I really liked using Opnsense before I switched to Openwrt and I always want to go back, because it was just so much more streamlined and easy to configure from a 'geek but not a network engineer' perspective.
I actually used to be a network engineer (many decades and career reinventions ago), and I appreciate that Mikrotik exposes all of the layers and connections between them.
But damn if Claude doesn’t always find something I got wrong in the config.
I do use Mikrotik for 2.5g switching though, zero complaints.
I absolutely loved vyos but after their repeated hostility towards non paying users I moved to mikrotik for my routers (already had mikrotik switches).
First they stopped publishing LTS images, telling users to build from source. So people built gh repos you could clone and build with actions. Then they stopped making lts builds reproducible and told free users to use the preview release. But the preview release doesn’t get security updates. So if there’s a vulnerability you’re SOL until the next release which could be months. Not really tenable for an edge device for me and I’m not very interested in being a user of a product that does not want me.
Isn’t it a subscription? And super expensive? $3k/year/device was the cheapest thing I saw on their site or $12k/5 years. 25k/year for an MSP. On a 5 year cycle we’d need to manage thousands on devices to make it more attractive than pfsense (cost wise).
I really wanted to try it a few years ago but they seem like just another tech company that only wants millionaire customers.
It's royal pain.
Basic router setup is not hard.
Port forward to expose web server? Can be done after heavy docs&forum reading, but hair pinning never worked for me.
I checked their WiFi AP. It's UI is hard even for senior software engineer.
In the end I got opnSense installed in VM on Proxmox and it's my router for last 2 years.
I'm using 2 Mikrotik switches with their simplified UI to operate VLANs. Zero complains.
I still like them but it's usually not the networking knowledge that trips people up.
But with an LLM it’s easier than falling off a log and the command line means it can do everything easily.
I remember I misconfigured it, and can't really Google around or ask LLM to fix it, because my Internet is down now.
Of course I can use Internet on my phone, but it's not really convenient setup to ask LLM to troubleshoot the router.
I ended up switching to back to Ubiquti.
Ubiquiti sucks nowadays but Microtik is simply insane.
It's a great educational experience I would recommend to anyone interested in networking. Grab debian, grab a system with 2 NICs, make it your router/firewall/gateway.
There's also some really good value hardware that you can install OpenWRT on, I picked up a Xiaomi AP for like $30, flashed OpenWRT and it works great.
It's not something I've tried but I imagine you could actually expose FRR directly and configure through that
You might be interested in https://www.youtube.com/watch?v=KZpJvpm1Ris (title: Does UniFi FINALLY support IPv6 Properly? State of IPv6 with Unifi Network v9). The video is about a year old, but from the looks it's not terribly outdated.
I like ipv6 but I have been thinking of disabling it lately because i cannot technically setup static leases with ipv6 like i can with ipv4, so technically the ipv6 for one of the services in my homelab could change and cause problems.
But also, even if a device doens't support DHCPv6 IA_NA, you can always also have a DNS server in your network, and you can bind SLAAC-generated Android addresses to simple subdomains. This seems better than DHCP anyway.
Chose them partly to reduce dependence on US tech too.
and it's cheap and fully customized, and powerful
For me when building homelab, customizability maybe important
And all that is available from tiny home/SMB office devices starting at 40 USD, through dedicated wireless point to point bridges, to a big Ethernet switches/routers with 400G interfaces.
Of course it doesn't compete with high end NVIDIA Spectrum-X, Cisco or Juniper, but is good enough for 95% of networks.
It works great as a plug&play home/office router, too thanks to Quick Set, but for more sophisticated setups, a knowledge of networking protocols is mandatory. The more fancy setup, the more in-depth knowledge is required. Just like with programming and all stuff in life.
That was always what was holding me back from switching off a 2.5GbE / WiFi 7 UniFi stack.
Is that representative of using mikrotik hardware?
Nice detailed write-up too, and great explanatory diagrams.
Other routers:
1. OpenWRT is a bit too basic, and updates are hit-and-miss.
2. Ubiquity is a bit too Apple-like, with its reliance on apps and integration.
3. Cisco is WAY too expensive, and too complicated to set up without having Cisco experience.
I'm now eagerly waiting for them to release the next generation of their routers. hEX Pro looks perfect to replace my aging CCR2004.
I tried Mikrotik and the UI was wasting so much of my time, it was worth switching to Netgate, for me.
Scripts on RouterOS handle network checks for failover, DNS updates, etc.
I got that Mikrotik home router 10 years ago and it is still performing great and handles most of the use cases without issues.
UDM-SE it is for now.
They of course also have a DS-Lite AFTR gateways to tunnel the IPv4 traffic. In practice, everything works (for me at least).
step 2 throw the mikrotik in the garbage
The problem with this (assuming you meant a generic (x86?) system) is that you won't get wire speed processing (no hardware offloading), so network performance will suffer. Unless you buy a dedicated router board (which is typically ARM), like say something like the Banana Pi BPI-R4. But installing/configuring a standard distro line Debian for such a board would be a PITA (if you want to do it right and set it up optimally), so you're better off using a dedicated router distro like OpenWRT.
Modern CPUs are quite capable. The problem isn’t performance, but power use.
Setting up a router with Debian is really kind of overstated in terms of difficulty, and frankly half of everything wraps Debian anyways (looking at you Ubiquiti/Vyatta), so why learn someone else's half cooked configuration language when you can just do the real thing.
Setting up Debian as a router might take some effort but it's certainly not difficult. Enabling routing is a single line config change, install dhcpd and you're almost there. I'd certainly recommend the router OS.
I route my home 10G internet on a ThinkCentre Tiny with a 2018 Core i3-8100T.
real-world OPNsense (yea, that's not Debian) benchmarks show that even low-power Xeon-D (Skylake-D SoC) with 10G SFP+ NICs:
pushed 9.40 Gbps in plain router mode
7 Gbps with packet filtering and no rules
7 Gbps in NAT mode
5.5 Gbps with a default ruleset
- 1x10GB SFP+
- 1x2.5GB Ethernet w/ PoE In
- 7x1GB Ethernet w/ PoE In/Out
- powered by DC jack, 2 pin, or PoE
- Similarly sized to 220 × 125 × 22 mm (approx 8.66 × 4.92 × 0.86 inches)
- Max power consumption 150W
- Silent
- ~$300
And I can replace my RB5009Upr+S+IN.
But the actual router? That lives on NixOS in a virtual machine on a generic amd64 machine that also runs NixOS. I'd rather design / perfect my config once in terms of iproute2/nftables and know I can swap in any hardware. I've got a full policy routing setup that let's me define for each device what Internet horizon it can see (along with what other local devices it can access).
I also find the sibling comment mentioning routing speed to be funny. I've found that unencrypted gigabit WAN is easily handled by most amd64 computers. 10Gb LAN->LAN hairpin routing seems fine. But the big thing is doing gigabit OpenVPN/wireguard, which off the shelf hardware-accelerated routers just don't really do unless you spend a lot. (coupled with the above policy routing, I can configure things like this particular web browsing VM goes out a data center IP that rotates every day).
Wireless APs are similarly handled by hostapd on NixOS, driving (mostly) Mikrotik MiniPCIe Wifi cards. I don't really need the latest and greatest wifi gen, rather I just need them to work and be as maintenance-free as possible.
All my life I have had to make do with junk I could get at Walmart or Best Buy. Got tired of it. Cheap plastic shit that would inevitably shit the bed just a month or two after the warranty expired. I'm not making the IT helpdesk wages I was making 30 years ago... fuck that. While I couldn't spend ten grand on home networking, I could finally spend a little something on hardware that's not total shit.
Got the CCR2004-16G-2S+. No stupid ONTs, either, just the SFP+ modules. Plenty of backup 1gig ports if I get a third WAN (thinking about getting Starlink on standby). Actual proper fucking APs that do seamless handoff (hell, one of them probably covers the footprint of my house, if I'm honest). And no more shitty GUI configuration pages. I just shell in and band out commands on the command line (at least three times til I get the syntax right, but who gives a fuck).
Wish they made something with 3 or 4 SFP+ ports that I didn't need a rack for.
I installed openwrt and then installed it on many mikrotik routers over the years. to be honest, not so much traditional routers (wifi+4-5 ports) but on ~10 port mikrotik switches for my infra.
i've since been switching to zyxel gs1900-* switches (up to 48 ports with openwrt)
Yes, I think it connects to the mikrotik cloud automatically to setup a usable external DNS to your device, but I believe access to it is turned off by default unless you turn it on.
https://manual.mikrotik.com/docs/network-management/cloud/
https://help.mikrotik.com/docs/spaces/ROS/pages/197984280/Ba...
The other reason is for updates, it can check periodically for firmware updates.
But outside of that it's not like streaming network telemetry or anything like that.
I’m sorry what? You can run openwrt on that line of switches?
I just have a 24-port one at home running stock firmware…
It's possible to run v3 under Wine on a Mac but it was kind of clunky. V4 is much nicer (also runs on Linux too).
[1]: https://llmstxt.org/
Ctrl+F in it for 'command provides information about RF' and the same on https://help.mikrotik.com/docs/spaces/ROS/pages/224559120/Wi...
But the degree of inconvenience is up to the beholder.
One way is using something like ChatGPT with a pocket supercomputer. It's tedious to get technical stuff done that way, but it can be accomplished. (For those who haven't tried it, the bot can be spooky-good at correctly interpreting cell phone photos of technical text on a screen. This saves a ton of transcription effort.)
Another way is with the phone's hotspot and just about any random computer. Laptops have wifi. Wifi dongles for desktops can be stupid-cheap, and it's useful to have a few scattered around. It works fine (unless one's hotspot data is ridiculously-limited, I guess).
The best way is probably by being prepared with procedural fall-back methods. Keep backups of your configs, and keep the recovery instructions with them. Test recovery; an untested backup isn't necessarily better than having no backup at all. Keep as many historical versions as one wishes, and automate it because procrastination is part of the human condition.
After that: When the router's config gets stuffed up, then the backups and the restore instructions are right there. Just pick an appropriate backup and use it.
We should probably be keeping backups of important stuff like that even in the absence of LLMs. Humans also err, and stuff does break, and what seemed obvious six months ago when we were in the groove might be impossible to remember when we need it today.
I have like a ~15y history of working with their stuff.
like:
https://www.amazon.com/dp/B0978TF8K3/
https://openwrt.org/toh/zyxel/gs1900-24hp_a1
or:
The one thing that it frustratingly does not support that I wish it did is stateless bgp with ipv4 prefixes. It cant offload an ipv4 packet with an ipv6 next hop.
https://mikrotik.com/product/crs305_1g_4s_in
Each of the mikrotik switches even comes with a block diagram linked on its product page, showing for instance a single 1.3 Gbps between its L2 switch chip and its CPU.
https://cdn.mikrotik.com/web-assets/product_files/CRS305-1G-...
Oh the stories I could tell you.
I once had to work with a guy who didnt believe in dynamic routing.
Which wouldn't have been too bad but he also didnt believe in route summarisation or any kind of planning. So the network was carved up into random unpredictable blocks.
He sadly did very much believe in Mikrotik scripting, and so had scripts setup to disable/enable static routes based on very broad (often external) reachability criteria.
Whenever googles DNS had an issue the whole network would start flapping scripts like nobodies business.
The network was a large wisp with more than 600 pops and more than 10000 customers on its various services.
That engineer had been gainfully employed by multiple ISPs over 20 years.
Other things: switch port management and bridging, proxy ARP, and so on.
Not sure what you mean about bridging either.
I've never had a use for proxy ARP so wouldn't know, but I do know it's possible to enable it. If not on the GUI, at least on the CLI.
I wouldn’t suggest anyone buy something like this today for home use, but 6 years ago when I purchased it there were not many 10GbE routers to be had period, let alone for the price ($300 roughly if I remember correctly ). Even considering the cost of power over the last 6 years, I still come out much cheaper than if I’d bought something more power efficient new at the time.
It draws around 15W idle, going up to 30W at full bore. For comparison, the 10G rental router from the ISP drew 15W no matter what (idle or full bore routing). This is with a 10GBase-T SFP+ which probably influences the power draw vs using a DAC.
It has no issues maxing out at 9.4 Gbit routing with a single or multiple streams of iperf3 with your basic IPv4 NAT+some port forwards config.
