Building an ARM64 home server the hard way(jforberg.se) |
Building an ARM64 home server the hard way(jforberg.se) |
The only ARM specific thing here is probably the need to use a DTB.
This just shows that manual Linux installation on random ARM board is not more complex than on x86_64. Perhaps even simpler, since you're just extracting a pre-made rootfs instead of using a package manager during installation.
For real though, what the author did is much harder than downloading and booting an official OS image from Pine. The article also documents all the successful steps and skips any missteps or debugging, making the process look very simple (not a criticism, I thought it was an excellent read). Maybe those missteps took place during previous projects, but suffice to say that you don’t make booting a non-standard image look easy without expending significant effort, at some time.
Anyway, I didn't say the things you're criticizing my reply for.
If I go through the article and list arm64 SBC specific pieces, it's just copying the booltloader from the manjaro image for rockpro64 and maybe different names of files in /dev for some block devices.
Even configuring U-Boot is done the same way you do it on regular Arch, if you use extlinux on x86_64, which many do/did.
EDIT: Probably yes. I see this term appears in the article.
The RockPro64 is a good board with lots of expandability. I run NetBSD/aarch64eb on one to build all of NetBSD's pkgsrc packages (26,000). It performs well with an m.2 NVMe, and has been rock solid.
Of course, for anyone using the RockPro64, if you plan to do lots of processor intensive work like compiling, you'll either need a very large heat sink (no Flirc cases for these, unfortunately) or you'll need active cooling. Without good cooling, it'll throttle.
Currently I'm running a home server on EPYC 3251 mini-ITX board, which I use to route 1GbE WAN and 10GbE LAN, serve as a NAS, and run a bunch of services all without it breaking a sweat, and leaving plenty of headroom shall I want to run more stuff there. It sits on my desk in a small-ish cubic Supermicro chassis and barely makes any noise beyond the normal HDD screeching. And it's an entry-level server-oriented board so I have proper LOM without having to throw in an IPKVM.
I would fancy an ARMv8 machine - just for fun of it (and possibly better performance per watt) - but I think I can't get anything comparable from a RPi-level hardware. But the next "step" I see when searching for ARM servers are those fan-screaming behemoths you put in a rack in a proper server room, which is something I dread for a homelab, as I don't have a dedicated room for it. I've had a pleasure of WfH involving setting up some PowerEdges in my living room, was fun but extremely noisy. So I wonder, where are the middle grounds?
It's a middle ground between Raspberry Pi and a 128 core they sell to cloud providers. For the money, you can probably get more work done with an amd64 workstation, unless you're paying someone to generate your electricity by riding a bicycle or something. (Cooling and power matter to cloud-scale datacenters, but not really for one computer in a room that you use to generate your income.)
I do wish that alternative boards had better OS support (especially the Rockchip ones, which tend to have weird kernel builds, etc),
A few weeks ago I bought an used intel nuc7 with a 7th gen core i5… for 150€.
It came with a 120gb ssd, 4gb ram and a power brick.
I still don’t see the value in this SBCs used as home servers.
If you think the price is high, I would point out that the SSD I used cost €200 new when I purchased it back in mid 2022. A used 120 GB SSD by contrast can be had for maybe €10 which alone would explain the difference in cost.
Now if 120 GB is enough for your application, that's a good value so more power to you.
My laptop is also a single board computer, technically, so what?
Though there are reasons to specifically want an ARM64 machine for builds, etc.
I think the general approach is very good and could probably be used for the VisionFive 2 RISC-V SBC as well.
I've been impatient for ARM64 hardware to become easy to use for home servers. I've got an RPi 4 (at retail prices, no less!) and it is quite good but I want more options. The previous stories I've read of RockPro64 have been much worse, it was nice to read this went relatively easily.
I run arch Linux arm on mine and it’s a fantastic little device. I wonder if these boards are way faster or just more of the same. I guess the pcie expansion makes this more extensible.
The slight problem with the deservedly often-recommended RP4 is that for most people it's so hard to come by it effectively doesn't exist.
No, you can't, unless you know of some source of retail priced Raspberry Pi 4s.
In some basic tests (compiling, ffmpeg), the Pi 4 and the Rock Pro 64 are within a small percentage of difference in performance.
I'm looking forward to someone making a NAS board on the new RK3588 since that has enough connectivity for everything I want.
I recommend using an SSD and not an SD card though.
My point was not that I was looking for a guide, but I have the feeling that subject is totally ignored by most of the sbc-arm community.
I take the risk of being victim of burglary as a when, not an if. I totally don't want anyone to have easy access to my data. If an SBC is used for server purpose and not for embedded/domotic use, it will likely contain data and/or secrets.
After about a month I had a barely working uboot built from unpatched official sources.
After two months I still didn't have a bootable kernel built from unpatched official sources.
- with power regulator drivers, the board powers itself off while booting
- without power regulator drivers, it boots the kernel, but there's no power to usb, ethernet and wifi.
What I learned: To stay away from Rockchip.
I did the same thing with a nanomimal http server to serve static content and maybe dynamic in the future: a noscript/basic (x)html http server for maps (which uses openstreet map tiles), which does provide proper map display in links2, with a font not too big, and with harmless html tables.
Configured the "server" to restart everything if something is detected missing (you know, cron with SH scripts and certainly not bash scripts).
It has been running for years. I never had to modify the code of my smtp server, yet (and I run IPv4 and native IPv6 provided by default to millions of clients by my ISP, I think it has been the case for more than a decade, may be wrong about this one though). I am kind of surprise it was not already pown by some trashy hackers.
The main issue: spamhaus block lists, they are hostile to all self-hosted people and they don't provide a irc server, or a non blocked email to be removed from their lists (which are unfortunately used by too many open source related companies/project, which is a mistake). Basically, they force ppl to use one of google/apple super heavy javascripted web engine (no better than the default security checks from cloudflare). Yes, those ppl are seriously worse than spam itself, hope they will fix that (they are a shaddy swiss-andoran company...).
Did you know you cannot send an email to redhat(IBM now) people using an ipv6 smtp? yeah...
And it is coming: I'll move everything to a similar RISC-V mini-computer because I am aware of the super toxic IP tied to arm64 ISA (same for x86_64), that will be the first step, the 2nd step will be to hand compile (=assembly programming with near Zero-SDK) all of them and forget this C syntax too complex and those horribly massive and complex compilers, not stable on the long run (thanks ISO, gcc extensions and c++). And with all that, I would not be surprise to port to 64bits RISC-V assembly a minimal IPv6 stack... and maybe more.
Seems silly to save $25 on the slower CPU and half the ram.
Orange Pi 5 16GB RK3588S (8 Core 64 Bit, 2.4GHz Frequency), PCIE Module External WiFi+BT,SSD Gigabit Ethernet Single Board Computer,Run Android Debian OS (M.2 PCIe2.0!)
https://www.aliexpress.com/store/group/OPI-5/1553371_4000000...
( via https://news.ycombinator.com/item?id=33739176 )
Review:
"""
"Orange Pi 5 Review – Powerful, No WiFi" https://jamesachambers.com/orange-pi-5-review/
Pros:
- 4 GB and 8 GB RAM variants cost under $100
- M.2 slot supports high speed NVMe storage
- RAM options from 4 GB all the way up to 32 GB available
Cons
- No WiFi or Bluetooth included (requires either adapter for the M.2 slot or a USB adapter to get WiFi/Bluetooth capabilities)
- No eMMC option
- PCIe speeds are limited to 500MB/s (PCIe 2.0, benchmarks show closer to 250MB/s write or PCIe 1.0 performance) — this is slower than SATA3
"""
And if you'd need stability once, you could just set Archive on a specific day as package mirror on your cache server.
In my own anecdotal experience of running a hobby server on Arch for several years, I haven't experienced anything to make me think the distro is unsuitable for server work.
I think we all use the software we choose to use in order to use the software we choose to use. When we build a cluster, it's often done in order to build a cluster.
Eg. if major postgresql update comes, you'll have to upgrade your DB cluster very soon. If major update to some program requires configuration changes, or if scripting language has deprecations that you've ignored for years, etc. you'll run into trouble, too.
I've been running a few Arch Linux servers for ~5 years and it's been quite pleasant. Being able to use the latest features in various programs or scripting languages is a very nice benefit.
Well… at least its what I’ve wish I could say.
Truth is I’m using manjaro (arch based) on a similar board and then one day after an upgrade they just decided to migrate from eth0 to the current naming scheme based on nic driver. Had to plug in a monitor and keyboard to fix the situation. Home stuff so its all good in my case.
Allow me to correct that for you.
There is nothing wrong with spamhaus. They provide one of the best anti-spam options amongst all the commercial providers.
Spamhaus have many lists, I suspect the one you are referring to is the PBL, in their words "DNSBL database of end-user IP address ranges which should not be delivering unauthenticated SMTP email to any Internet mail server except those provided for specifically by an ISP for that customer's use.".
We are in 2023, I think it is beyond any sort of doubt by now that a significant proportion of spam and phishing mails originates from home internet connections because people can't be bothered to keep their computers up to date and virus free, so they become part of a botnet.
So the fact of the matter is that even if Spamhaus PBL did not exist, someone else (or the MX operators themselves) would very soon fill their place by blocking the very same ranges.
Added to which, most home ISPs don't even provide reverse DNS ... so again, even if Spamhaus PBL did not exist, you would likely STILL find yourself being blocked by other measures that most sensible sysadmins implement on their servers.
Hell, many home ISPs just block outbound port 25 these days anyway !
Spamhaus provides a way to be removed from this list, but does not provide an IRC server, only an horrible web javascript only chat, they should fix that. Ofc, they provide an email to request removal from their block list... which is using their block lists.
Since spamhaus is "shadily" hidden in andore and switzerland, my lawyer cannot do much, but I guess I should go after the sys admins using without grey listing those block lists in EU/US but I haven't needed too yet, since there is most of the time either somebody with a smtp server not using blocklists (not even grey listing) or even an irc server.
From a technical point of view, and specific to my ISP in my country (did not check the other ISPs), putting all domestic ranges of my ISP in their block list is text book abusive... spamhaus is doing a really, really, bad job. But I keep that for court if I need too, I may go to EU regulatory orgs directly though, well only if I am pissed off enough (and that's very hard).
> "If you want to make spamhaus remove your IP from their block list, you must engage in a chat working only with google/apple javascript browsers (I am a noscript/basic (x)html user)."
Amazing that I've been on the internet for several decades and never once had my shit jacked (due to a modern GUI browser or otherwise.) The way people like grandparent commenter make it sound, the split second you use a modern browser, you'll be pwned...
Edit: https://news.ycombinator.com/item?id=34700126
> webkit/blink/geeko are financed (and steered) by the same people: blackrock/vanguard = apple = alphabet(=google) = microsoft = starbucks = etc.
loooooooooooooooool
For starters, nobody is ever forced to use a web browser with email. I'm OK with the fact that pine will parse some of the HTML so I don't see all the silly tags in most email, but it will never follow a link, at least.
If your IPv4 and/or IPv6 is on a Spamhaus list and you can't get it / them removed, likely because you're in a pool of residential IPs, and likely in part because you can't control the PTR, then you can always smarthost through any reasonable provider.
I've been self-hosting email for a quarter of a century, and I'd never blame anyone else if I tried to send email from a residential pool of IPs and it didn't work.
Not sure what this has to do with setting up a nice little ARM server, besides your observation that the ARM architecture is licensed, but here we are :)
Those guys are bad, really bad. Hope they grow up and improve.
Yeah, once I have finished or I am more advanced on other projects, I'll get rid of those pesky arm64 with that toxic IP (that said it is the same for x86_64). I'll re-use first my C code as a stepping stone to perform the jump. One more step towards real digital freedom.
You'll just have your upgrade dance clustered into a single event once every N years, instead of spread over randomly, based on major releases of SW your server depends on.
Not that you know of, anyway.
By adding communication between the OS and the bootloader it's possible to implement redundant updates for whole partitions (specifically A/B-updates with a boot counter). U-Boot supports this (depending on the state of the vendor-provided fork better or worse), and Tow-Boot seems to be based on U-Boot.
https://systemd.io/AUTOMATIC_BOOT_ASSESSMENT/
https://www.gnu.org/software/grub/manual/grub/html_node/fall...
https://www.gnu.org/software/grub/manual/legacy/Booting-fall...
[1]: Minimum about 23 years, from personal use in creating a product with A/B-root partitions.
https://morefine.com/products/morefine-s500-mini-pc
The FAQ question reads a bit funny:
"Q19:Support ECC RAM？
YES. S500+ Support ECC RAM，
But compatibility requirements are higher, and cannot be used casually."
Also, these AsRock products:
I currently have a single 16 core/64 GB ECC Ryzen tower with ZFS that I'd like to break into a ZFS ARM ECC NFS server with a separate small form factor Ryzen or ARM compute cluster.
Most problems with btrfs is that it fails to mount in many situations and there's no automated fsck to fix it, requiring manual intervention; in fact running fsck on btrfs is considered a very bad thing. This makes it OK for desktop, but not at all suitable for headless servers and other unsupervised machines.
Speaking of bad design, F2FS for example, a filesystem designed for flash drives, keeps both primary and backup superblocks in the same flash erase block. If that block gets corrupted the entire fs is lost.
"you're not correct about Spamhaus and being forced to use Google / Apple"
What I meant is that you weren't necessarily correct about Spamhaus, nor correct about being forced to use Google / Apple (which I thought was a reference to the fact that 98% of the world use Google's browsers and Gmail and/or Apple's Safari and/or Mail).
I see now you were referring to using a mainstream browser to communicate with Spamhaus. Yes, that's uncool. And yes, I wholly agree that the email address to request unblocking should not be filtered like it is.
Sometimes we worry so much about the symptom that we forget about the problem. Perhaps it'd be worthwhile to just ask someone else to forward an email requesting removal to Spamhaus' removal address.
Spamhaus is doing a really bad job. They just need to grow up and improve.
So far I'm pretty impressed with my Rk3588 board, 8gb ram, and dual 2.5gbe.
It's a bit more than half as fast as my quad core/8 thread xeon desktop, but that's abotu where I'd expect one of the cheaper x86 NUC/Mini PC/Thin clients to land.
What happens? I don't know... fsck would run automatically on reboot, fix the errros and recover part of the last file written in /lost+found/
But why would a server abruptly shut down 1000 times?? I already said btrfs is good enough for desktops and other interactive/supervised systems. And data loss on servers is recovered from backups, not journals!
XFS is known to be extremely robust for servers - and it's journaled. Most servers nowadays are SSD.
