Cloud in a Bottle: making self-hosting accessible to everyone(cloudinabottle.org) |
Cloud in a Bottle: making self-hosting accessible to everyone(cloudinabottle.org) |
I picked it a few years ago for much the same reason as anyone else (sick of this shit), and it's main attraction to me was it being a turnkey solution that focus security so i wasn't as likely to completely fuck everything important up.
I think a "simple" installation package would be very attractive to most people, and while cosmos has a lot of growing to do in order to be that, I'm still pretty happy with it.
I'm even more happy to see more in this space, because I'm just at the point of "do i stick with cosmos or do i start rolling it all myself" and god the leap is terrifying if you're thinking of exposing to the internet. I get it, but at the same time it feels like you're always one missed config away from a potential nightmare, and that's after the "welp guess this is my second job" problem.
- You both need to trigger the potential OSS contributors, with references to Debian, Ubuntu maybe, and appeal to big donors, which is another topic entirely,
- And ALSO have a design that triggers the broader audiences and therefore, very "turnkey", very packaged, very "Comment like and subscribe", one-click install on your NAS, etc. Privacy is then a characteristic but services are the real features.
What am I missing.
WTF? I’m selling you a car with no brakes but it slows down reasonably fast?
That’s not an issue tho, I want each application to be a separate login, so I can have one pointed to a different subdomain or domain not all in one login and location.
What a load of bull.
Then I decided to painfully break out of vendor lock-in, built my own NAS with FreeNAS which was all the rage (still attached to a vendor, but at least I could control the hardware). This was before Docker really took off - the plugins were installed in FreeBSD jails. It worked for a while, but again, plugins could be woefully out of date and broken, upgrading the OS was a pain and would break all your plugins.
Through the 10+ years of this... I now follow two simple rules: - separate your network storage and application hosting. Yes, it complicates things a little bit with NFS setups but it is a forcing function for a more resilient setup - use the docker container provided by the maintainers or LinuxServer.io. Vendor maintained wrappers can be initially helpful but end up as a form of tech debt when they're abandoned or neutered - especially when they're freemium and the company starts looking to squeeze.
The quiet part of self-hosting is that it shifts the responsibility of security to the user. I'm sure cloudinabottle has more sensible defaults than provided docker containers, or builds in reverse proxies etc, but I doubt they're offering to take on liability for data loss or breaches.
The problem I see you'll face are two-fold: 1. Projects like this have the power users self-select away from it and those who need a lot of hand-holding to self-select into it. If I know how to configure and deploy a docker container already I have no incentive to contribute. ReadyNAS got around this by having folks charge for their plugins (that were just OSS projects packaged up for their OS) but then it ruins the appeal, they likely only got away with it because they were only a couple of bucks and you were already locked in to the ecosystem via the hardware. 2. If the hosted aspect actually makes you money, required for the project to be continued long term, capitalism will ensure that incumbent cloud providers take that away from you by offering ready-made instances at your cost-price, like they did with redis, elasticsearch etc.
Note that these are only issues if you're motivated by making a profit, or even a cost-neutral project. But if this is a charitable exercise backed by an entity that doesn't need the money, I wish you all the best - more things that let people dip their toes in are good! The cloud is all rent-seeking subscriptions and even with the recent supply shortages: fast, small, low-power compute is incredibly cheap these days, especially old business/mini machines.
- after selecting your observability apps, every _other_ app could automatically configure centralized logging, its own dashboard of metrics, etc
- the host OS could ask the user simply for the backup destination (S3 url creds, attached local disk, SMB share, etc) and every app is again automatically configured with backups
- the host could have a control panel to roll back apps between versions (maybe it's FS snapshots or something)
- everyone can use containers, but if the project went through the extra effort to make each app its own VM instead, that could be worth something
- if there was a companion Android/iOS app for management and (importantly) alerting?
---
That doesn't really solve the monetization issue you described, but I do think you can still get powerusers on board if you handle even harder problems than they'd want to take on themselves. Or...harder problems than they'd risk taking on themselves if their family uses what's being hosted, for example. Less tolerance for downtime
Edit: formatting
https://cloudinabottle.org/docs/setup/shared_homeserver.html
There's no reason this couldn't be made as simple as double clicking a .exe and then the whole thing gets setup.
If you want to message me, find my email on https://qbix.com/about
This is just rose tinted vision of the past. None of the software of the past had the capability, reliability or robustness of modern sass.
Regardless, I agree that Docker has made self-hosting stuff pretty easy, and arguably even easier than it was in the pre-cloud days.
Docker has poor tooling for network level security between the containers, has issues with different runtimes per container, etc.
It is just a bit primitive if you want to expose multiple services to the internet on the same server. One of those apps will get compromised and then all the others will follow.
If you want a decent self hosted server, ask your frontier LLM agent of choice to configure kubernetes (on something like k3s) with mandatory userns mapping so nothing runs as root on the host, default deny firewall so inter-container communication is as locked down as possible, and if your router supports, set up VLANs so none of the containers can access your other devices on the LAN. Use something like backrest to handle backups, alertmanager and Grafana for monitoring, Keel for auto updates. Also consider separating ingress for public and internal services and use Tailscale with split DNS to acces the internal entrypoint. Set up Crowdsec as WAF and subscribe to their free blocklists to filter out bots. Ask it to set this all up using Ansible, so it can be maintained.
While this would be extremely time consuming to set up and maintain by hand, an agent can do and test it in a few hours.
Could you be more specific about that?
> One of those apps will get compromised and then all the others will follow...
Per their security docs, containers are rootless but I don't see anything about VLAN isolation.
Aha so LLMs have better tooling for network level security, cloudboy?
golang.
So no fail over/redundancy? I think at minimum it should be two machines, so one can break. That is one of the important features of cloud applications as far as I'm concerned.
You don't have to deal with the problem of a single computer breaking.
> So no fail over/redundancy? I think at minimum it should be two machines, so one can break.
It's clear this product is intended for personal use.
I guess you have a couch in your home. This couch can seat friends/family. Yet you don't have a spare one?
This happens with making many small to medium level resource hungary apps, for personal usage, run on one VPS which is not a heady duty machine costing heavy duty money. I have tried this on similar machines 2GB RAM and 1-2 VCPU and after 2-3 apps it starts swaying like a drunkard in slow motion even in that SSH session. But at least this one seems to be better than something like PikaPods, in combo pricing regard (I am not saying one is generally better over the other), where the pricing is rather per app.
Yeah, that’s not what cloud computing is.
I'm a little surprised that there's just one backup app, instead of each app coming with its own backup sidecar. To backup Immich, for instance, requires coordinating between its four different programs. I guess I'll read more about how that backup app works too ha
Sometimes it means "rented virtual resources", sometimes it means "stack of intertwined services", sometimes it means "SaaS", sometimes it means "alternatives to SaaS"...
Here it seems to be the latter.
bewcloud could be a lean alternative for home users if that's something you'd like to offer.
I haven't heard any complaints about NextCloud, they've added some enterprise features, but all in a positive way, from what I can tell, not detracting from home and personal use.
Instant spin ups and restarts
Easy interfaces ssh web cli AI agent
Integration aplenty
Trillion dollar companies try to extract more and more from us, the only response is for us to develop alternative solutions and continuously chip away at their strengths- what made them monopolies in the first place.
I had rather good experience using agents to setup and deploy own cloud, Jellyfin and co. I experimented also getting a VPS, just giving the LLM the ssh keys and having at it. It’s pretty good.
If the AI stays afloat, then I think the future lies somewhere there. Not too sure about the security side of the things …
> just giving the LLM the ssh keys and having at it. It’s pretty good.
Whereas, it you let the AI go free on a server, you mostly have no idea what it did and where, so it's quite hard to audit and even harder to try and fix things.
So, unless relying on a declarative paradigm like NixOS does, I'm not ready to give AI the keys to my castles
I'm fine with giving a "smart" self hosted LLM like qwen 3.8-flash-next Q8 access to it to do things in opencode.
Don't give it the keys to any castle but give it access to one specific containerized thing...
Bits missing from cloud being global CDN (selfhosted), routing, regional sharding, etc.
The docs mention cloudinabottle.toml but the app repos all have openhost.toml. Did you folks change the name of the platform?
Also, can you point to how the unified auth works? Couldn't find anything in the manifest spec.
It proxies requests from the user to installed apps, and adds a header 'is-owner'. Apps have to be built to respect that header - some apps do this by running their own little proxy/sidecar that will detect that header and inject saved credentials.
Try it out and see
Containerisation was supposed to mean host anywhere, literally redeploy a complex set of services and their connections on any cloud provider at the drop of a hat. That is what a f**ing shipping container is, you rock up to any port in the world and the same container is lifted by any crane onto any ship or truck.
Amazon, Google, Microsoft - these people have sold vendor lock-in to the very segment of tech society that should have been fighting it tooth and nail. 20 years ago you ftp'd some PHP files and it just worked, now you have to figure out 20 IAM profiles and shit your files across buckets because everything must be 1000000x scalable out of the box.
No joke I was building a static firebase site the other day and just wanted a simple way to password protect it (like http auth) for staging so my client could see it - of all the useless shit firebase tries to lock you into they can't even do something as simple as basic http auth without a shit show and spinning up a cloud run instance or something to check a static password.
Yes we 100% need fix hosting, everything should be portable, simple, generic.
I'm thinking from the "20 years ago" it is the first one, so your experience is even more valuable for those younger persons reading it because you've seen how things were before the vendor-lock-in era began.
I ask because I've been seeing an increasing trend of people who've self-taught themselves Linux/BSD things, grew up in the cloud-native era, and have now become increasing dissatisfied with the "Do everything in one vendor cloud" operations model that they initially learned. Many are now going the opposite direction and deciding to self host their own projects at much lower cost and much less vendor lock-in.
https://github.com/search?q=%22cloudinabottle.toml%22&type=i...
If for some reason your services can't speak an already common language, I think it's on you to add those shims into your own project.
Let's assume that they didn't have these container restrictions and could just use an ordinary docker-compose.yml, they'd still probably want to use something like OAuth2.0 for centralised login which a lot of these services won't have.
What would be better, making a massive PR to add support for it, or opening an issue to discuss it with the maintainer first?
The biggest contributor to the project in the last 3mo is andrewlaack... and we can also just take a guess that "adl" in adl-collab is AnDrewLaak.
If yes, regardless of who made the issues, that seems like terrible architecture/design there...
This line makes me think you are running a spam campaign and andrew just isn't part of it
Many projects disagree with you on this one. It is quite common to have a policy that forbids PRs without an issue or discussion first.
Meta and Google can spam their unsolicited ads to a billion users but regular people aren't allowed to promote their own projects on niche forums? This is bs. You're the problem here. Self-promoting your harmful values. You're the spammer.
Is there something similar in progress for a "tech company tech stack"? As in, rather than trying to assemble your own custom infrastructure by finding an RPC system, permissions/group manager, credential management, storage, database, service discovery, job management, monitoring etc. one "opinionated" stack that you can easily deploy, as long as you're OK accepting their choices, with all these parts already bundled and wired up.
Currently, a lot of this space is docker compose based and simply inaccessible to so many who may otherwise be interested. Whether or not it's this project or another, I hope something gathers enough steam soon to truly break the barriers to entry.
And yeah AWS and the likes made for big money spenders with gigantic DevOps teams, not one person handling everything.
We at DollarDeploy were advocating that for 95% of your usage needs you don't need anything but one server and if outgrow it just switch to a bigger one.
I think one cloud in a bottle should be solved differently - apps deployed independently but additional thin layer for MDM - so logins and core data is at one place.
Imagine if your self hosted wordpress, outline, nextcloud and etc share same logins and access levels, bootstrapped with same information without the need to type it for every app.
It often seems like presenting the image of being autonomous to our peers seems more important than actually achieving it.
It takes a lot of energy to replicate what AWS and friends have done once we factor in concerns like the passage of time and entropy. I've never been able to keep a media/NAS appliance alive for much longer than 3-4 years. Inevitably, there is some kind of catastrophic event and I have to start all over with new vendors, etc. Even without any data loss this gets old. I promise you get tired of this after a while. It might seem like that would never happen and then one day it does.
You can get so much more done if you can actually afford to use the cloud. I have a really hard time believing that many people are genuinely winning on total cost of ownership with self hosting.
I gave up sync and availability for sensitive data to at-rest encryption. Mundane photos/documents can stay on Google Drive, for now. I wish there's a better solution.
Looking at your hosted page, I see no reference to backups. I see in your docs reference to backup, but you should definitely offer some sort of turnkey, appropriately-marked-up backup solution. Configuration difficulty is what kills self-hosting on the front end, updates make it hard over time, but when it all dies and then I have no backup is when I give up.
I can't find any reference to disk size limits anywhere. Can I host Immich on this? Can I host my media server? What will it cost? Surely not $10/month for my media collection. I'm not a hoarder and mostly have stuff ripped from my personally-owned media, which limits the size, and you're still not hosting that for $10/month. I see you have Jellyfin in the set of apps but I don't know what it will cost. The CPU & RAM limits I understand from the deploy page.
Oh, and I'm specifically referring to your hosted service here.
See docs on how this works: https://cloudinabottle.org/docs/how_it_works/data.html#the-a...
I'm working on a project with similar goals but some different design decisions - instead of the typical containers I'm going for thin-client apps that hit a common data layer built on a fully serverless architecture. Not ready for production quite yet, but for those interested: https://starkeep.app/
I'm surprised there's no such thing as a self-hosted deployment spec. An opinionated docker-compose file (or similar) with well defined inputs, outputs, and requirements that works for the average use-case.
The "workflow automation" approach is a lot of resource overhead and broker-config maintainance, but my kingdom to be able to arbitrarily compose apps' states, eg commutimemap.com + Craigslist.
0. https://www.cloudfoundry.org/technology/open-service-broker-...
1. https://github.com/gavmor/immich-concourse-resource
2. n8n.io
Since we have AI now, I think exe.dev's pitch is better: they provide Linux VMs, a web proxy, and integrations. You can ask the AI to build whatever web app you like. I'd love to have exe.dev, but running on my Mac Mini.
I'm working on exactly this bit (with some more tuning for messaging AI agents)
https://github.com/gofixpoint/amika
Doesn't run on your Mac Mini yet, but should get that enabled either this week or next
If you sign up (free) I'll send you an email when the "Mac Mini mode + full OSS self-hosting" is live
I’m trying to self-host as much of my stuff as possible, but don’t have time to tinker like I used to. Hence updates, backups (etc) fall by the wayside in favour of “set and forget”.
Are you planning to offer alternative solutions for this, e.g. pointing a wildcard A record at the server running the instance? I'd rather not run a publicly accessible DNS server.
If you're building a personal cloud, you also want your AI connected to it, but probably with some degree of management. Aperture already provides a fair amount in this area, and probably with a connector for Cloud in a Bottle, could give your AI safe access.
I did a few pitches related to this at Nasscom Startup warehouse and IITM RTBI. Though this was welcomed by the panel members but it never took off due to lack of capital for such a startup during those times in India. I had to abandon and focus on my day job.
These barriers are high enough, so that common people can't create/host their own sites. Only skilled enthusiasts or professionals can afford having their own site.
An unique IPv6 may be theoretically assigned to each end device and thus solve the accessibility problem for locally-hosted servers, but I doubt internet providers do this in practice and I am afraid that many providers don't support IPv6 at all.
Only on HN, the general public don't care, they don't even know the name of their web browser.
Fwiw I've started to see a shift in many non-tech circles I'm a part of. Especially when you take the time to explain the problems. Many do care, they just aren't made aware of the issues.
A lumberjack may also argue GP don't care about environment laws if they had timbernews.
Self hosting implies having hardware and software knowledge to set it up. I agree, having a meta package that would set the system up as systemd would be an even lower barrier. But then again, there are podman commands that do this for you today anyway.
Self hosting could also imply buying a device that just does this with an additional management interface for updates. This could be build around a nuc/nas with above package. It still will require some knowledge to keep it maintained.
Then, you could offer maintaining them as a service again. Maybe there is value in that.
Update: apparently there is https://freedombox.org/ which is actually an appliance for this. Although it is more privacy focused and I don’t see a productivity or photo suite on it.
Because it requires people to also have knowledge about the ins and outs of docker and docker compose. If we are talking about web applications it is yet another layer that has been added over the years. It used to be that you could fairly easily host most things on a old fashioned lamp host (I am talking decades ago) and all you had to know was basic file transferring.
Of course, this was on shared webhosts in a time when VPSes were not really a thing yet or affordable. But, even on a VPS setting up a LAMP stack is relatively straightforward.
Once you add modern dockerized application to that mix you are now still looking at some sort of ingress to do the reverse proxy bit. So while it likely will not be Apache or nginx there will be some sort of layer there in addition to now having to setup docker properly (rootless and all that) and then also making sure your docker compose setup is in order.
It is not difficult *once you know how to* but it is another layer of knowledge and experience people need to acquire. That does make it less accessible for novices.
Building the equivalent of Google Photos locally wasn't that hard 10 years ago, given enough tolerance for speed and indexing lag issues. Trying to do the same today will cost two arms and leg and require a lot more overall knowledge of what to compose to get the same result.
The dust will settle and we should go back to a nearer gap in I hope a few years, until the. it might be a glacier age for self-hosting.
You could argue it’s much easier on the software side, because self hosting Immich is much easier thanks for AI assistance with setting it up. If you can live without the built in facial recognition feature you don’t need particularly beefy hardware.
I find myself taking on far more ambitious self hosted projects thanks to Claude. Extending home assistant in various ways, deploying a “self hosted Spotify”, making sure everything is fully accessible via Tailscale out of the range of the wifi and so on. Of course the general population isn’t really aware or interested in most of this.
Only Gen Z and later are starting to care and willing to spend the time and effort to self host. All the Gen X and Y people that I know of, including many that are very intelligent and accomplished in their field, literally have no idea how to operate a computer outside of watching youtube, checking emails, and opening Word documents. Ads? They hate it but have no idea how to disable them. iPhone running out of storage and iCloud says I can solve that for $2.99/mo? They would be like IDK what that is but sure I'll pay that $2.99 if that means I can record more videos of my kids.
I really hate this situation, and I try my best to help out the folks around me who I see are getting ripped off (either they don't know or they know but think that's what it is). But still, it's really hard to communicate or teach those people when the skill of operating a computer has become intuition and human nature to me. I really can't put these things into words on the fly that would help them understand :(
I'm not sure what you are seeing, but what I'm seeing is widespread teach illiteracy in Gen Z outside of their smartphone. Gen Z and younger mostly grew up with smartphones as their primary devices and with little contact with a laptop or desktop computer (and even then confined 99% to browser usage).
I can't quite confirm that. My younger siblings' classmates (still high school students, born >=2010, non tech average people) don't give a shit on these topics and behave identical to the Gen X and Y described.
But maybe that was kind of related to times and the place – where I live people needed to be a bit more savvy, so thought like "why you buy an iPhone if Xiaomi can do the same 4x cheaper; if you chose the right model you may even have a good camera too" was popular even in non-tech circles.
But yeah, now that hardware is a bit more of a commodity and such tradeoff might give you a bit more adware with a bit less convenience, maybe it's different. But the next generation's understanding of computer/phone as something else than a magical black box is gone.
Yeah, you’ll need a lot of supporting evidence for this claim. From my personal and professional experience, this doesn’t track. And given how online the younger generations are, it tracks even less. I think there might be a bias towards tech bubbles in that statement.
Also while Imbue is in some sense an AI startup it does have a very different ethos from a lot of tech companies: https://imbue.com/about
I haven't touched my nextcloud installation in maybe 9 months and I've been running it for about 1.5 years.
When there is a problem it's now very easy to fix, I just point an agent at it. It syncs to my phone, my laptop, and several other people's as we use it for a small business.
It's not perfect but neither were any of the commercial things we'd tried before either.
If you are an individual and you want to store a media library, you'll go broke instantly.
I have roughly 200TB of storage capacity, 100TB used, in my NAS. That would be $2300/mo on S3 before even getting to access fees and network egress.
There's no way you are consuming this much content actively. You could store all of it in S3 deep archive for $200/m (100TB) and maintain a tiny fraction in the standard resource pool.
IPv6 works exactly how you describe, that's the whole point of it. No NAT, just IP addresses. Some gaming consoles use it for peer to peer multiplayer so it is in demand even by normies. One of the problems is competition is stifled in the ISP market, by monopolies and ridiculously long contracts.
Samsung web browser uses chromium under the hood
And it's not to fault Immich, on the contrary, Photos is one of the rare Google property where the team is constantly pushing the envelope.
I could combine it with a bunch of external tools to make do, but it would be crazy clunky or would need full sync with Adobe Cloud for instance for the editing part, while doing all the scanning and categorizing locally.
On AI assist...I'm still pausing a lot when t comes to home projects that will be constantly talking to the outside and hold the stuff that is the most private to me and my whole family and friends. It helps me as a research tool, but due to my own restrictions it won't be magic.
Totally understand this part. I didn’t mean to give Claude code full access to locally hosted environments but rather copy back and forth in your local shell with help from Claude (error messages, helping to plan out the installation etc.)
Setting up and maintaining a server is easier today than it was decades ago, A LAMP at this time meant a lot of manual setup, IaC wasn't a thing, reproducibility wasn't a goal, versioning was confidential, documentation was scarce and often outdated, out of the box security was lower, ties to the OS were higher. Managing virtual hosts was clunky, updating OS/PHP was risky. I would not go back for anything.
> having to setup docker properly (rootless and all that)
Rootless being the proper way to setup Docker is a highly controversial take. You will mostly get added complexity and a false sentiment of security from it.
You are speeding past my point by a mile or two. With a shared webhost you get a ready lamp stack. In the late 90s early 2000s you could fairly easily get a wide array of software running that way. All you had to do is create a database in the hosters control panel, upload the package and you'd be on your way. Later in the 2000s one click installers came along for popular packages making it even easier. In fact, they are still around these days making it extremely easy to install a wide array of lamp based applications to these days.
There are very few options these days to get started as easily these days with modern non lamp tech stacks. The only thing I am familiar with (other than the product presented to us in this post) is pikapods, [which only offers a limited selection of available applications](https://www.pikapods.com/apps). There are a few others, also catalog based, though they more seem to focus on deploying on other platforms for you. Other than that you start to quickly move to more complex hosting solutions aimed at business and scaling. Or platforms that are cloud platforms with all the added complexity to navigate and figure out before you can deploy docker containers. Not to mention ridiculous situation that a few of the ones I know about have pivoted to being "agentic compute providers" whatever that means and certainly will confuse a novice. Which means that the "practical" advice given is often to "just" set up a VPS. And yes, many VPS providers will provide an image with docker enabled, but that is just the start.
> Rootless being the proper way to setup Docker is a highly controversial take. You will mostly get added complexity and a false sentiment of security from it.
That, in fact, is another layer of complexity people will then have to figure out. Most information I am aware of these days does claim that it is better to run containers rootless or at the very least make sure the user in the container is non root. The fact that we are both convinced of the opposite tells you how confusing it must be for a novice.
The overall point I am trying to make isn't even about the exact details. It is that for hosting something these days there is much more required surrounding knowledge required before you can get started in most cases.
The world would be quite miserable if "perfectly legal" would be what everyone optimizes for.
what you will find is that this is what corporations optimizes for. Therefore, you as a person, cannot compete with the corp.
That's why you need to become like a corp, or make the rules "socially acceptable" the same as "legal" (which is the best option).
And you can still live quite comfortably.
They're probably jealous that they never had the guts to do it themselves and prefer to rely on friends in high places or daddy big media to promote their shit for them.
Standing behind your work is good and wholesome. Fear of self-promotion is weakness.
I often talk myself up and promote my work. Especially face-to-face. If your work actually measures up to the talk then you have nothing to apologize for. You're actually educating people and doing them a service.
Don't let yourself be bunched up with snake oil salesmen who didn't actually do the work. If you did the work and nobody noticed, then it is your duty to talk it up.
The fact that a maker can't also be a seller is dumb. You can switch modes. You have to do whatever it takes to fulfill your mission.