Oxide Cloud Computer. No Cables. No Assembly. Just Cloud(oxide.computer) |
Oxide Cloud Computer. No Cables. No Assembly. Just Cloud(oxide.computer) |
AMD Pro CPU with SKINIT and SEV
AMD OpenSIL + OSS coreboot firmware
Motherboard with Infineon 9672 (or newer) TPM for DRTM secure launch
ECC memory
Add-on modules for OcuLink [2] (external PCIe) and Nitrokey (2FA, HSM) with OSS Rust firmware [3]
OS support for QubesOS (with Oxide management VM) or Oxide custom OS
High-integrity client workstation within Oxide manufacturing supply chain(s)
Customer local admin of Oxide rack
Customer remote admin of Oxide rack, with mutual attestation
Oxide remote troubleshooting of customer Oxide rack, with mutual attestation
[1] Framework, https://oxide.computer/podcasts/oxide-and-friends/1632642. Lenovo and other OEMs may follow Framework's lead.
[2] OcuLink expansion module, https://community.frame.work/t/oculink-expansion-bay-module/...
[3] Nitrokey Rust firmware, https://github.com/Nitrokey/nitrokey-3-firmware
Who should be interested in this product? Does it make sense to compare this to AWS, Google Cloud or Azure?
"Cloud computing" style systems are nice in some ways - you can just ask a computer to give you some virtual computers and virtual storage and it gives it to you. Whoever owns them can put quotas or pricing or whatever on you, but you can self-serve, and you don't have to care about replacing DIMMs or NVMe sticks or whatever.
Having some random American megacorp host things in a datacenter is good for some people, bad for others. You might not want to be in their legal jurisdiction, or you're legally not allowed to, or you just don't want to, or their prices for your volume are too high, or you don't want to be locked in to whatever future bad choices they make.
So, Oxide made racks of machines you can buy, plug in, and then have a cloud-style (virtual machine, virtual storage, virtual network) system at home.
I really really don't understand what is hard to understand.
I mean, not every single person on HN is a 10x developer that knows 300 programming languages known to man and 45 more known only to catgirls.
I'm a daytime Windows admin, this isn't stuff I normally work with, especially because it's targeted at a specific stack of things that I don't touch.
I really really don't understand what is hard to understand.
edit: And it seems like it's aimed at companies that don't want to pay cloud margins, but don't (yet) have the expertise to set up a production-worthy Kubernetes (or similar) cluster from scratch. An opinionated appliance vs DIY approach.
Feel like there is a larger potential customer base there but it also seems like they would lose the edge they built by owning the full rack. (I.e. integrating with customer TORs and network fabric is a nightmare.)
Unplugging the Debugger - Live and postmortem debugging in a remote system - Matt Keeter [1]
The talk was at the Open Source Firmware Conference.
Pretty cool look into how their system works under the hood.
See https://learn.microsoft.com/azure/operator-nexus/azure-opera...
AIUI Microsoft will ask you to buy several racks worth of (oem?) server gear and switch fabric, configure it to load up their version of kubernetes, and then leave you to run whatever workloads you like (or they approve of? Not sure) with the hook being that you can manage it all from azure.
Pointed strongly at telcos, and I imagine that you cant get this without spending at least a quarter mil on hardware. Plus whatever azure fees there are? I wonder how many msft expect to sell, especially as telcos with spare cash are like unicorns.
https://console-preview.oxide.computer/
https://github.com/oxidecomputer/console
I also wrote up a blog post walking through me setting up a server by hand: https://steveklabnik.com/writing/using-the-oxide-console
You can also use the API, there's a terraform provider, etc.
[0] https://github.com/oxidecomputer/hubris - fascinating little kernel
That's pretty cool! The design language is a nice touch for sure.
[0] https://oxide.computer/blog/the-cloud-computer
[1] https://oxide-and-friends.transistor.fm/episodes/launching-t...
Bunch of discussion then: https://news.ycombinator.com/item?id=38023891
We are not just buying servers, we are buying 'cloud'.
Why its on HN? Maybe somebody discovered it for the first time?
I don't get the platform side.
What guest OS's does it support? Can you create "bare-metal" applications that run in some kind of container on it? Does this resemble a re-invented ESXi?
How does the performance and redundancy of their storage layer compare to something like GRAID?
What is the total overhead (in terms of cores, memory) of the management layer with Oxide (incl. block storage, vmm, etc.)?
I'm seriously impressed at how much they improved the on prem experience
> Contact Sales
Nope, hard pass. If you don't list your prices on your website I'm never going to be a customer.
Is the a market for these?
Sun did announced "NeWS is Free (only $1000)" which was kind of a kick in the balls and abuse of the word "free".
"Free (only $1000)" discussion on usenet:
https://groups.google.com/g/comp.windows.news/c/-G1l_372AP8/...
https://groups.google.com/g/comp.windows.news/c/YtZzYMZ587M/...
https://groups.google.com/g/comp.org.sug/c/IXcXK7hKgUQ/m/nf8...
A couple of my flames about the situation:
https://www.donhopkins.com/home/archive/NeWS/Explanation.txt
https://www.donhopkins.com/home/archive/NeWS/sevans.txt
And to cleanse the palette, DSRH's epic "Sun Deskset == Roy Lichtenstein Painting on your Bedroom Wall" flame (David SH Rosenthal was one of the original authors of NeWS, with James Gosling, and also wrote the X11 ICCCM):
I think a modern take on NeWS style system would use Webassembly style system with capability based access rather then PostScript and no security. Basically a modern browser with Canves/WebGL and without a lot of other stuff a browser does.
Would have been a interesting alternative to the Wayland approach.
Kind of sad that Scott McNealy didn't have the balls to open it up. Having some real competition to X in the 90s would have been a cool. Specially if Sun had pushed it at least somewhat.
dell famously doesn't have one price either.
They didn't just "tell him," they laughed their heads off at him. Advertising prices and selling direct Just Was Not Done. Until it was, leaving a trail of bankrupt competitors that looked like that road in Iraq.
I also realized I never read usernames.
Just to be clear, though this did seem to get cleared up below, the level of abstraction you're working with on an Oxide rack is VMs, not k8s. If you wanted to run your own k8s on top, you could.
> because they’re not able to set it up themselves sounds insane to me
It is not about ability. It's about quality, and what you want to spend time on vs what you want to spend money on. (and of course time is money...)
There's a lot that goes into building and maintaining a private cloud. Some would prefer to build it themselves, some would prefer to focus on their core business and buy something that works well out of the box.
> What’s the plan when it breaks? Send the server back?
Building a robust product is very important to us, but so is supporting it. If something breaks, you contact support, and it gets sorted.
An advantage here is because we have created almost everything ourselves, under the same roof, we have fantastic insight into how the system works. No pointing the blame at some other vendor's firmware!
If that illusion breaks and you need to get into the weeds in the same way you do with self-hosted k8s, then the value proposition of their product goes poof. I'm just speculating, of course.
Yeah that was my reference point - cluster at home. Breaks often & hard and usually end up wiping it. Good for home use but I’d not want to rely on it for prod
> buying an out of the box k8s because they’re not able to set it up themselves
You basically get Networking, Compute and Storage. Everything else you have to build on top.
You will also get stuff like Terraform provider, SDKs and so on.
> I can
I guess if you are millionaire you could.
It is a practice cluster though so I’m not exactly careful in my experimenting
Enterprise CIO doesn’t want a hobby project (attempting to cobble together internal cloud orchestration and infra), they want to be able to show immediate business value. You charge what the market will bear. I’ve seen many companies with thousands of employees and spending millions, even tens of millions a month, on public cloud providers and just flail, unable to get to steady state post transformation (even after years of trying). This is made for those folks, especially with Broadcom having VMware self inflict harm on itself with recent strategy decisions.
“Write check. Cloud up.”
(no affiliation)
It's going to depend on how well they manage to pull off the magic trick of "little or no configuration and maintenance required". If things start breaking in hard to diagnose ways, it's going to be just another broken appliance that requires expensive maintenance, and companies will be questioning why they didn't DIY it in the first place.
You are also paying for a bunch of stuff you don't need. Most people just don't need to hot swap a CPU or turn these single socket 128 core machines into a gigantic 4096 machine either.
Simply moving virtual machine off and restarting or replacing a sled is enough for the waste majority of use-cases.
This is still pretty much commodity single socket server platforms, just with more sane and open firmware and a sane open source software stack.
For example "3.2TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 with carrier" is $3,301.65 each, and you'd need 10 of those to match the Oxide storage spec -- already above the $30k total price you quoted. Similarly, "128GB LRDIMM, 3200MT/s, Quad Rank" was $3,384.79 each, and you'd need 8 of those to reach the 1TiB of memory per server Oxide provides.
With just the RAM and SSD cost quoted by Dell, I get to $60k per server (x16 = $960k), which isn't counting CPU, power, or networking.
I agree these costs are way way way higher than what I'd expect for consumer RAM or SSD, but I think if Oxide is charging in line with Dell they should be asking at least $1MM for that hardware. (At least compared to Dell's list prices -- I don't purchase enterprise hardware either so I don't know how much discounting is typical)
Edit: the specific Dell server model I was working off of for configuration was called "PowerEdge R6515 Rack Server", since it was one of the few I found that allowed selecting the exact same AMD EPYC CPU model that Oxide uses [1]
[1]: https://www.dell.com/en-us/shop/dell-poweredge-servers/power...
That’s the pricing for people who don’t know to ask for real pricing — it’s an absolute joke. I don’t know how might extra margin gets captured here, but it’s a lot.
Even in teeny tiny volumes, Dell will give something closer to real pricing, and a decent heuristic is that it’s at least 2x cheaper.
This is a real SSD. Dell likely buys this brand and others:
https://www.serversupply.com/SSD/NVMe/30.72TB/KIOXIA/KCMY1RU...
Yes, that is almost an order of magnitude cheaper per TiB. If you buy from a sketchier vendor, you’ll get all the way to 10x :)
Its almost all open software, that helps a lot. They add a minimal amount firmware, rather then the many, many million lines of firmware that is usually around. And most of the stuff they added is Rust on a micro-kernel. (Check out the talk I linked top level to see some of their low-level debugging infrastructure).
To bad they can't (yet) get open firmware in the NIC, the SSDs and some of those other places (Time for an Oxide like company that makes P4 driven NICs). But nobody else can really offer that either.
The only real issue for them is that Illumos is the host OS. Its open source and stable of course, and has good debugging tools. But in terms of industry experience, the amount of people with deep knowlage of the system are harder to find compared to Linux.
The of course also add some complex software on top that will have to work properly, moving VMs, distributed storage and so on.
Full DIY is pretty damn hard, you need a serious team to pull that off. The Dell VxRail/VMWare is the more reasonable competition. I think VMWare going full Broadcom mode will make them more interesting. Buying into that ecosystem isn't that appealing right now.