Ask HN: Which self-hosted solutions are you using? I came across Perkeep[1] project today. I know some other self-hosted solutions like Rocket chat[2](Slack alternative), Redmine[3](Project management) etc. Curious what other self-hosted are there and which one you are using? [1]: https://github.com/perkeep/perkeep [2]: https://github.com/RocketChat/Rocket.Chat [3]: http://www.redmine.org/ |
- Cost. My needs are small so in practice I can stay in the free tier of most services.
- Availability/Reliability work. All the maintenance burden would now fall over me: data backups, network failures, etc.
Could you comment on these points, from your experience? (also whatever other concerns you may want to point out about self-hosting services)
As for cost, I run everything on a $5/month DO box, and it's only using half the ram and barely any CPU. You could run it on an even cheaper VM or even a tiny physical box like a Raspberry. Backups are just a matter of using a tool that encrypts and syncs to some offsite.
Cost wise, my self hosted operational expenses are -way- less expensive than any cloud offerings, let alone signing up for every SaaS/PaaS service I want to try out or actively use. Most offer self-hosted/self install for free, charging for hosting on their infra. A single, reliable small VPS runs around $3-$5/mo at the least. I have almost 30 VMs (not all powered on at one time) on my own two e-bay purchased servers in HA. Also around a third of those VMs require more RAM or storage than the low costs VMs. So just having those VMs would cost around $200 a month. Though if I used cloud based, I likely wouldn't have nearly as many VMs.
Upfront costs for the server + drives was ~$500. I also have a NAS that was around $500 with drives. So all in one time costs around $1000. I've now used those items going on 6 years, replacing a few hard drives here and there.
Backup wise, the hypervisor backs up images and snapshots to the NAS. I backup the entire NAS to Backblaze B2 at a cost ~ $100 a year. S3 would be around $750/yr alone.
Self-hosted: Capex: $1100 (hardware, some licensing items) Opex: $300/yr (hardware replacement, backup costs, external necessities [domain, Voip trunk, etc] )
Cloud hosted: Capex: $0? Opex (Lets say half the VMs, 15x$5) : $1700/yr
In terms of availability and reliability to make it work; it certainly does take some time and effort to maintain. I am responsible for all the updates on the OS and software of course,and I can't blame anyone else for failures. Again, I like this level of control and responsibility. I can update on my schedule for down time or when needed for security fixes. I'm not waiting around for some one else to apply the fixes or inconvenience of unplanned downtime for it. I'm not saying my stack is more reliable than AWS or Azure, but I've taken steps to minimize issues. I'm still at the mercy of my one ISP line and electrical outages of course.
What concerns me about self hosting: You're responsible for everything. Security most of all, people seem to skimp on because it is difficult or gets in the way. You can't make a 100% secure system anywhere, but if you're self-hosting you absolutely need to work WITH security in mind and not sabotaging for the sake of convenience or 'just getting it working'.
It's not a set it up and forget it situation. You still have to do updates and occasionally fix things when they break. Reducing the amount of time and any barriers to this is key in getting it done. Personally, I use orchestration and automation tools to help.
Interoperability with other services/APIs can sometimes be finicky to get right, especially behind NATted situations. Sometimes not worth the hassle.
Some ISPs block certain well known ports. Of course, NATting services that are not well documented, also sucks sometimes. ----
TL;DR: I like the control that self hosted gives me. In my case it's cheaper than AWS/VPSes. If you do self host, keep your stuff updated and as secure as you possibly can.
- Gitlab and Gitlab CI
- Zulip chat(moved from Mattermost)
- Metabase for data visualization(will be adding InfluxDB soon)
- Self Hosted Sentry(error reporting): Sentry is linked with Gitlab and Zulip for alerts
- Taiga: project management tool
- OpenEDX: a beast :)
We are Python(Django) based ourselves and love that we can tinker with many of these tools as they are too Django based.
At personal level, I use:
- Mailinabox: Self hosted mail
- Nextcloud: For file syncing, contact management, calendar and tasks, notes.
I have also looked at Perkeep several times(even set it up two times) but couldn't include it in workflow. Especially mobile file syncing.
* Youtrack for bug/project tracking (made by the awesome guys pf JetBrains)
* Matrix for chat (only some internal rooms on the private instance, the users stay on the main official Matrix)
* Koken for photo galleries
At some point I'd also like to add:
* a map tile server for openstreetmaps, to remove any external dependency on Google/Apple/any maps
* a jitsi server
* a secure file storage/file sharing solution
- Apache Tika + Tesseract for OCR of mails - i hate physical paper
- SOLR to index the output of the above mentioned data
- Imaginary (https://github.com/h2non/imaginary) for image pre-processing of the scanned mails / documents. Its much more lightweight than imagemagick.
- Openhab2 for home-automation
* SSH
* OpenVpn
At home, started as needed:
* Samba for backups (dedicated NAS)
* Gitlab (VM on desktop)
I don't need much more. I'd like to merge all of this into my NAS, but the 14 hdd box draws 70W idle. I'm planning to use a rockpro64 instead to do all that. Maybe then I'll find other useful things.
Oh, and I have a RPi1 as a 3d printing and mqtt server.
I had a Mattermost server a couple of times in the last few years but I really don't need it.
It's not always perfect but after I set it up it has just kept running. Browser add-on and phone app work decently.
- synthing for file sync
- miniflux for RSS
- WordPress for friends/family websites
- prosody for XMPP
- wallabag for bookmarks
- centralized rsyslog & collectd
- radicale for calendar/tasks/contacts sync
- gammu for sms sending
I’ve been running a live stream and vlog all year on it, now carrying it around in a crate with a UPS, and posting design videos that follow scalable patterns with minimal tools.
Nobody can ban me from the Internet, now.
I also recorded bootstrapping videos and most of the development via screencasts and all on GitHub.
Kinda pitchy I know, but I sell nothing except services and only state what I’ve done, in fact—I believe in it.
and much more
I also felt that self-hosting could bring security issues to the table where nothing was before... but of course the huge savings in other fronts might compensate the extra work. In any case, that's a good looking setup you have there!
One of my favourite self-hosted apps is WikiJs (https://wiki.js.org/). It takes a git repository of markdown files and turns it into an editable wiki and syncs back changes to the git repository.
I also use InfluxDB with Chronograf and Telegraf (https://www.influxdata.com) to collect and analyse logs. It's not quite as full featured as Elastic or Prometheus but is easy to use, rock-solid and nice to look at. Plus they sent me a free pair of socks (all the way to NZ) for filling in a survey! :-)
And to help manage all the Docker images I spin up and down, I use Portainer - https://portainer.io/
Forgot to add that just recently I've started to run Apache Guacamole to give me remote access to my local LAN while I'm not at home. This is so great - HTTPS access to RDP, VNC, and SSH sessions. https://guacamole.apache.org/
alias dcup='docker-compose up -d'
alias dcrestart='docker-compose down && dcup'
alias dcpull='docker-compose pull'
alias dcpullup='docker-compose pull && docker-compose up -d'Then one day someone decided they needed "reports". We ditched our working system and paid a bunch of money for the Atlasssian suite. To this day I still miss our simple, functional, and free setup.
Bamboo was so bad that we eventually gave up on it and switched back to Jenkins. Seriously Bamboo is one of my most hated software products of all time.
We tolerate Confluence but the fact that the JIRA and Confluence have their own, annoying flavor of markdown, which is different from Bitbucket's flavor annoys me to no end.
Bitbucket (which was called something else before) does have a few nice features but nowadays there are options better than gitolite alone. JIRA has those pretty reports that someone wanted and that are never accurate.
yay.
How do they justify this? It trips me up every single working day of my life. The products look identical so all I see is a text box on a white site with blue accents and I inevitably start writing the wrong markdown.
We also use RedMine (running on GKE) and that was a breeze to setup and deploy.
So, while today I'm not interested in the 'lists', when I come back looking for x,y,z I might be.
I think I might even be starting to prefer it to git/git(lab|hub).
[1]: https://www.fossil-scm.org/index.html/doc/trunk/www/index.wi...
- Syncthing + my own FileShelf[0] as a web interface, instead Dropbox;
- Miniflux, a web RSS reader;
- IPython notebooks;
- Customized Jekyll for my blog + comments powered by self-hosted Isso;
- postfix/dovecot + Roundcube for the web mail;
- ejabberd (mostly to have notifications from my VPS);
- goaccess for the web server statistics, portainer and a couple of FCGI scripts for monitoring and deployment;
I am still not satisfied with my setup completely and there are many other things I'd like to tinker with (e.g. DokuWiki, Grafana/Prometheus).
sandstorm.io looks very promising, but the last time I was put off by its desire to take over my VPS (maybe I will bite the bullet and get another dedicated VPS for it).
* VMware VSphere/ESXi
* JIRA / Confluence (managing tickets and wiki)
* FreeNAS (server facing file server duties)
* Synology (user facing file serving duties)
* Zimbra (mail server)
* pfSense (gateway, dhcp, dns, vlans)
* SIPP (SIP endpoint testing)
* Elastix (Asterisk PBX/VoIP Server)
* Windows Server (DNS, DHCP, file serving)
* Openfire (XMPP/Jabber server)
* NAKIVO (VMware backup server)
* Zabbix (SNMP/IPMI/Server monitoring server - amazing product but a PITA to setup)
Thinking about trying Proxmox though once my VMware licenses come up for renewal.
As an aside the most trouble free, zero maintenance server in my home lab is the Windows Server. Updates itself without filling up /boot and killing itself (Ubuntu I’m looking at you...). It takes up hardly any resources on the Hypervisor and just runs and runs. Haven't had to login to it in years.
Really wish Microsoft brought that rock solid ethos to Windows 10. Their server products are absolutely incredible.
Any drawbacks?
Pros for Zimbra are:
It supports Exchange Web Services and ActiveSync so you get complete feature parity for setting up Macs/iOS and other Exchange supporting devices. Makes it as easy setting up users as a full O365 setup.
Licenses are pretty cheap, we're paying I think £400-500 for 25 users per annum.
The web interface is good and has very strong feature support for aliases, mail filtering (O365 filtering is abysmal).
Supports adding External mail IMAP/POP accounts so you can send/receive all your Gmail emails from Zimbra.
It is very standards compliant with excellent IMAP, CalDAV, SMTPS, POP3, EWS, ActiveSync support. No half baked support unlike IMAP on Exchange or Gmail which occasionally does something protocol hostile.
Cons:
Zimbra web interface is not as good as Gmail or O365. It's okay though and feature rich.
If maintaining your own deployment expect some hours put in setting up the usual stuff like SPF, DKIM, DMARC, LetsEncrypt.
Standard support is slow although they do eventually get the issue fixed.
I don't believe Zimbra supports anything like O365's Shared Mailbox feature.
Spam filtering is not great, certainly not compared to Gmail.
* Seafile (file hosting/synchronization/sharing/history)
* SoGo (a webmail that work with existing IMAP and SMTP servers and exposes an Exchange API)
* Matrix / Riot (matrix is a chat server, Riot is the web client)
* Jenkins 2 (Continuous Integration / Continuous Deployment)
* FreshRSS (RSS aggregator)
Authentication is managed by a single LDAP service (openldap).
I also plan to test/deploy:
* Peertube (video hosting)
* Mastodon (micro-blogging)
My next goal would be to distribute this stack on more than one server, in order to improve availability.
* GitLab (GitHub + CI/CD replacement)
* FreeNAS (Storage server)
* Nextcloud (Dropbox replacement)
* Syncthing (Dropbox replacement, Nextcloud did not work properly on my android phone)
* In progress: Kubernetes cluster (just for fun, RKE + rook.io)
* My own notes/wiki/task tracker I will release real soon now (TM)
Thinking about setting up Gitea + Drone.io, GitLab is just too heavyweight for just using it as git Hosting + CI/CD.
The rest worked great and fast (~5 minutes I'd say). I am currently looking into persistent volume solutions, Rook.io looks good but is still beta. Ingress is another thing I have to look into, probably just port forwarding from my router to one of the nodes running the controller.
The nodes are Ubuntu 16.04 with docker installed from the repos.
I had a related thought recently when trying out the SecureScuttlebutt social network: ssb seems like a format that could fill the intended use case of Perkeep (throwing all your stuff in a database) with the added advantages of (a) having broader applications today, (b) having a wider range of mature db clients and (c) having a well-functioning existing hosted ecosystem for cross-device syncing.
This isn't ssb's intended use case and I haven't tried it yet but I intend to.
Emby-server for my home media library.
APU OpenBSD router for my outer-most internet router at home.
Unbound LAN resolver with a number of upstream unbound instances at different VPS providers.
Libvirt hypervisor for personal projects.
Synology DS411slim NAS and one homemade with emby, I'm wanting to replace the homemade one with FreeNAS mini though because I'm not happy with the HW in it.
At work I've setup, or helped setup;
Owncloud for internal file sharing and to clients.
A couple of gitlab instances, one for internal dev stuff and one for a client.
A homemade password pusher.
In-house developed monitoring system.
Really this list could go on forever so I'm going to stop here. Having worked for 7 years at my current position with setting up various open source solutions.
Because people were sending so many passwords over e-mail or text messages so it was a necessity.
[1]: https://gogs.io/
[2]: https://www.mattermost.org/
I've got it on a linode and across all my computers.
Me, I’ve got one for all administrative stuff, one for my web WIP, one for my personal photos, etc...
And each doesn’t replicate on the same other servers, as you can choose by entry.
It's free, and by self-hosting it I can keep our firewall rules locked down on the DB.
* Dokuwiki - Nothing new but it just works for my personal company memo. Its simplicity is good.
* monit - I love the easy to understand syntax of the config to monitor servers. Can also monitor the freshness of SSL certs when these days everyone uses Let's Encrypt and they expire pretty quickly. And bought m/monit a few times and I get to manage multiple servers' monit from its dashboard. I used to run Prometheus + Grafana for making pretty graphs of server activities but while the set up wasn't so hard, I felt the whole thing was a bit overkill for just monitoring server health and I'm happy with m/monit's simpler graphs. (Also does down sampling and purges old data.)
* InspIRCd - Not using anymore but had it for internal IRC chat before switching to Mattermost.
Tried BitWarden recently for password vault and it seems like a solid one but the clients are not as versatile as Enpass, so, still looking around.
- Embed (Embed.ly replacement) [2]
- Nextcloud (Dropbox etc)
- foobar2000 (Spotify :P)
I’m yet to depend on any of them, but projects I’ve been eying up are:
- Airsonic as a Spotify replacement (FOSS iOS clients)
- Kodi as a smart tv/box replacement (had this for a while, worked great!)
- NextCloud to host files (generally allergic to php, but too scared of iOS binaries from China to trust the superior SeaFile)
Most of the on premise IT, with the exception of the Jenkins servers, are managed by a local devops company.
We are a small company with 7 employees.
Some of the things I use heavily:
* Nextcloud
* Bookstack
* Privatebin
* Mailu (although I'm slowly moving to a manual setup)
* Wallabag
* Selfoss
I'm also waiting for pixelfed, peertube and pleroma to mature a little more.
I'm moving some of my content away from hosted platforms to self-hosted ones, and looking forward to integrating DAT[1] and Gopher so people have (relatively) safe non-HTTPS options to access content.
It's harder than it seems, although a large part of the puzzle is aggregating the data from these sources and getting them into a format that works. I'm considering creating a lifestream generator using my self-hosted data, and then syndicating from there to various services.
[1] - https://datproject.org/
The first thing I knew something was wrong was when my subscription to BBC Good Food Magazine changed, and I could no longer email myself recipes in any meaningful format (I had to screenshot the page and share it).
A common one is when a service is bought out and goes to crap, or when a service just goes under (like ma.gnolia) and you lose everything.
Plex has also burnt me by not letting me access content on my own network when their online service goes down.
Facebook burnt me more times than I can count, but Cambridge Analytica was the last straw.
My favourite type of burn is the breach of a security company burn, like lastpass.
Atm I'm preparing to also selfhost a small shop frontend and a Peertube instance for my Mastodon instance. I also want to overhaul monitoring and log shipping to centralize it and maybe reinstall a wiki for keeping track of everything.
The Author also pulled WebSub support for no good reason.
Rook is doing OK for me so far, but be warned, the api is seemingly quite volatile at the moment
For ingress on bare metal (I assume that is what you are using), either nginx-ingress (for rest protocols) or metallb for arbitrary ones.
It has always annoyed me that almost everyone uses exchange or gmail. It's refreshing to see people use something else.
I run Gitea since the goal there seems to be more like Github while Gogs seems to aim more for a Gitlab-Light experience.
At the time when I compared to Gogs, I read about a period where the Gogs maintainer was off, and PR's were piling up. Not sure, but that may have been the reason to create Gitea fork at the time. The project has multiple maintainers, and when I compared activity (about 8 months ago), it was more active in terms of PR's and commits.
Most of the core stuff is the same, so for simple git repo hosting I don't think it matters.