Self hosted email continues to steeply decline(labs.ripe.net) |
Self hosted email continues to steeply decline(labs.ripe.net) |
Also had a couple of ui pages on our app to have full email functionality.
This also means custom integration into our existing ecosystem.
I used Gmail for Domains for a hot minute, but never really liked it very much. Soon I found Fastmail, and I've been using them ever since (so about 13 years now).
It was complicated to get security, DNSSEC, DKIM, DMARC right this way. I did it just under a year ago, around September 15, 2025.
I tried to follow and understand what I was doing, and keep good records on it. It was a complicated process, but everything was up and working at the end and I received email my server sent me. It felt good to be able use this simple script to send myself email:
#!/usr/bin/env python3
import smtplib
from email.message import EmailMessage
msg = EmailMessage()
msg["From"] = "(user's from address)"
msg["To"] = "(my gmail address)"
msg["Subject"] = "(a test subject)"
msg.set_content("(a test body)")
with smtplib.SMTP("127.0.0.1", 25, timeout=30) as smtp:
smtp.send_message(msg)
print("Message submitted successfully to local Postfix.")
I understand that as my server's reputation grows, it becomes very powerful to be able to email anyone without any intermediary except potential spam filters. I am very careful about what I put on my server.
Nevertheless, I had a recent experience that was a little sad for me. I wanted to forward some mail to my gmail address automatically, and used ChatGPT Sol 5.6 on medium thinking time to help me with the administration to do this. (I find extra high and pro thinking times are excessively long.)
I was very disappointed with the results, which included a very badly written regex for authentication. I'll write up my experience in an experience report. It was a big disappointment for me.
The burden on administrating one's own email is already very large, and it is disappointing that we can't really rely on frontier models to help with it.
EDIT: "I see no upsides to X" means "X is NOT good". (Responses suggest that this was not clear.)
"PRISM is a code name for a program under which the United States National Security Agency (NSA) collects internet communications from various U.S. internet companies. The program is also known by the SIGAD US-984XN. PRISM collects stored internet communications based on demands made to internet companies such as Google and Apple under Section 702 of the FISA Amendments Act of 2008 to turn over any data that match court-approved search terms."
https://en.wikipedia.org/wiki/PRISM_program
"Section 702 permits the Attorney General and the Director of National Intelligence to jointly authorize targeting of non-US persons reasonably believed to be located outside the United States.
By targeting, it is meant that US persons or persons located in the United States may not be the intended targets of the collection. The targeting must have as its object the acquisition of foreign intelligence information, as this is defined by FISA."
I remember when Gmail came around and everyone was more than happy to hand off their email to Google. No spam and ability to send and receive huge (for the time) attachments I think were the main things people wanted.
Not sure if that was before or after everyone was happy to hand off their web browser to Google.
NO, and I avoid both of 'em as much as I can. (Note clarifying edit to my initial comment above.)
TBH, I blame IT leadership. The easy choice is Google or Microsoft. "No one ever got fired for buying IBM" is now "No one ever got fired for subbing to Office." Someone needs to bite the bullet and stitch together an alternative array of services. They've cultivated their own monoculture by hitting the easy-button and forking over for Office365, etc.
Which if we're being honest, Microsoft and Google products are rarely best-in-class at any one thing that they do. Teams, SharePoint, Google Docs, someone somewhere is doing those things much better. Those competitors just don't necessarily have a whole ecosystem that makes it a big Easy Button for corporate IT to smash. And they sometimes charge as much by themselves as Office365 costs in whole. But I know one problem my own employers IT team struggles with a bit is putting out the constant fire of different teams deciding the Microsoft tool we pay for sucks and going rogue and trying to subscribe to something else. Dropbox was a big one, but Teams and Planner alternatives come up a lot too.
Not pretending to have all the answers but it doesn't seem to be a completely intractable problem.
If your email is Google or Microsoft, there are other people with self-hosted email who feel pressured to use their Google or Microsoft accounts to send email to you.
(Blog author) runs the email infrastructure measurement project at Live Direct Marketing
Google offers "mail merge in Gmail to send personalized email campaigns, newsletters, and announcements to a wide audience." [1]
Based on how much @gmail.com spam I used get within GMail I can only assume they treat mail from their internal accounts much more favourably then us dirty self hosters.
Don't listen to the naysayers.
The only way around this seems to be hosting with Google, Microsoft, or maybe Proton. Any other ideas?
And setting them up properly is not even that difficult nowadays
I self-hosted for years and I did all the standard things you're supposed to do, but I had tremendous problems with Gmail and Outlook. Other providers usually went through without issue.
Unfortunately this only talks about what DMARC policies are published by domains. I was hoping they will go into some analysis of how many mailboxes actually hard-enforce DMARC rules on ingress (as in, respect the DMARC policy as published by the sender's domain, p=reject or whatever else it might be)
I have been hesitant to enforce DMARC rules on my server. Partly because maintaining it was a pain (see opendmarc - maybe better with rspamd now), but also because on my test runs I saw plenty of legitimate incoming mail that simply would get rejected if I were to enforce DMARC on ingress.
I suspect big players like gmail have some kind of soft enforcement/reputation/machine learning approach to DMARC, but this is way too much work for a small time admin like me.
The only thing I think can compete with that is some kind of decentralized solution, also something I've thought about a bit. Like a Keyoxide-based[0] - or similar - setup that makes it virtually impossible for one to lose their online identity (since lost keys can be rotated out once one has - access to - enough proofs), and then that identity can be used - without fear - to route associated data, etc. Of course adoption of what could become an adjacent or new standard would be the next hurdle to overcome...
On the other side of the coin, what would happen if google bans you? That's certainly not unheard of.
I have been running Postfix for 22 years, from a big corporate installation to my own on a Raspberry Pi Zero. It just does its job.
I've self-hosted for about 2 months, and promptly switched to iCloud. Just because I've been spending too much time calling people and telling them to check their spam folders.
Recently our business moved to another domain. We had a guy who has spent over a month "warming up" traffic, so our new domain won't be tagged as spam by default.
I've written about it five years ago, still applies: https://schumacher.sh/2021/05/10/running-a-private-mail-serv...
However you do have to live with the occasional blacklist when you host on a VPS like I do. Some idiot in your IP range may get blacklisted for spam and then you get caught in it. It’s generally temporary.
And there are some corporate policies that will flat out reject emails from you which is fun.
I’ve probably spent less than 10 hours over the last 5 or 6 years on maintenance.
>And there are some corporate policies that will flat out reject emails from you which is fun.
This is what keeps me away from self-hosting it, though. As long as there's some possibility that my ability to deliver email could get nuked, all because someone else made a mistake, it's not worth it to me, especially since using / sending email is mandatory for doing most things. I guess I'm just not really willing to take that sort of risk
The problem lies with the big e-mail providers marking your e-mail as spam, even if you have everything configured right (MX, SPF, DKIM, DMARC, etc.) and you're not on any blacklists. They use their mysterious opaque algo to determine that your e-mail is spam and you have no idea what you could do to not get in peoples junk boxes.
And that’s not hard?
Running a modern mail server, a spam filter, a self-hosted webmail, and learning just enough about DNS and networking to get mail through: this is what’s not that hard.
Figuring out how to stay whitelisted, or accept ending up in people’s filters, asking them to whitelist you while your public IP gradually gains trust, is a job.
I just wish these big parties would give you a more formal way to make clear you're not a spammer.
Edit: typo
My self-hosted email filters too little spam, while GMail etc filter too much (e.g. important company correspondence leading to orders almost being lost because they landed in our GMail spam).
And yet, a quick search of "psychologists near me" in Google Maps gives me a list of professionals most of whose email addresses have MX records pointing to Google servers.
I've done it for decades, but I don't know much beyond the basics of email hosting tbh, however my understanding is that these days email coming from self-hosted servers carries a much higher spam/junk risk and so often automatically marked as spam.
Maybe there are ways to avoid this happening, but when a few clients all reported that they were not receiving my emails I didn't have much of a choice but to switch.
"Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting."
LLM-written content is now too common to be interesting and complaining about it does not make it stop, it just derails comment sections. The reptitive complaints are getting as annoying as the LLM-written content!
Why should your opinion be worth more than those who would like to retain a semblance of meaning in what we use to convey ideas? LLM writing is sheer trash. It comes off like a coke-addled car salesman who is making a pitch to VCs about his latest scientific discovery that will 200x their investment.
Poorly written LLM content is NOT a tangential annoyance. If this is too much, maybe you should simulate HN using
Generative Agents: Interactive Simulacra of Human Behavior https://arxiv.org/abs/2304.03442
I LOVE reading LLM content. I find that they tend to present things in a way that keeps my attention - something I have struggled with my whole life. I also use LLMs to assist with managing my own thoughts and helping make things coherent overall.
Sure, there is a LOT of slop out there, but the promotion of quality content is what crowd-sourced curation like HN tends towards. Much of the LLM assisted content here has a high signal to noise ratio, and is clearly a lot more involved than simply prompting "write an article on X" - despite what the haters say.
Kill me. With a spoon. Slowly.
I stopped reading at this point. I cannot emphasize how much I HATE this AI writing style.
I stopped here, anytime an author (or likely AI) feels the need to tell me they're being honest or "genuine" I don't need to read any further.
I'll assume honesty but the minute you explicitly tell me, you've probably got an angle that I'm too dumb to see, but I'm not dumb enough to keep reading.
Is this in the same vein as using — em dashes?
Are people with minimal regard to typography and style doomed to lower their standards because the masses can't even?
Everyone should aim to be this lazy. Everyone is trying to maximize their time. This is what software has always done. It helps people be lazy in some respect, and then, so far, we have always found new stuff to do with the time we freed up.
LLMs got decent at writing, and before long they will be better at write-ups than the average human writer and we won't look back.
Of course, this is contingent on your current goal: If you want to get better at writing, you should do the writing, just like you want to do the math, if you want to learn multiplication. Writing is fine as a tool just to get information across. It needn't be more than that for everyone.
Judging by people's response, that doesn't seem to be the case.
> and before long they will be better at write-ups than the average human writer and we won't look back.
I'll read articles written by them if and when that happens, but not before.
Delegating this task to automation serves to undermine all the masters and indeed strengthen the servants.
Short term this offload is good for people; long term it’ll lead to mental rot where people are hard pressed to put their ideas forward and give up in exasperation when they don’t have the assistance of their servants. Their ability to even make the proper requests to the servants may deteriorate enough that their existence may come to rely on the benevolence of their servants.
I hate LLM writing, but the economics checks out.
That's the thing, good writing is expensive, massively so, much more than people think.
Letting the machine do the writing however shows us that many people have no idea why writing is expensive, nor do they respect that perhaps it needs to be expensive. Communication is incredibly hard, very few master it. I'd even go so far as to say that clear, precise and readable communication is so hard that we do not have sufficient training data to build an LLM that can do it.
Letting a person write article, like this one, doesn't necessarily improve the quality, but it does show us that some thought and effort have at least been put into it. There was a cost and you deemed that cost worth it for conveying your message. Without the cost there's nothing stopping you from wasting peoples time.
If I ever go back to self-hosting email, I would do the same except I would maintain a whitelist of friendly mail servers to which I would send directly. Or maybe just relay emails to Google/Microsoft via SES and send the rest directly.
How do you know? I have seen blocks of ip addrs blocklisted from AWS due to others abusing nearby ip addrs
I'd argue that self-hosting is the first step. The second step is helping everyone you know get their own domain and host their email for them. If they don't trust you to not read their email on your server, point them to somebody else who offers mail hosting.
The more people we can convince the internet is something anyone can help build outside of the silos, the more likely it is to survive.
Email spam has made self hosting untenable.
They should also deal with the individual accounts etc, but thats the difficult problem.
You should self-host, so you can do the experiment for yourself. You might be surprised. You can always stop self-hosting if you don't like it.
But I self-host email for ~20 years, and I have not managed to get rid of Spam without not also getting false positives.
I first used SpamAssistant, in later years Rspamd, but I feel like they are just not good enough. Also I find Rspamd config pretty incomprehensible.
Hosted email like also does not solve this, e.g. GMail filters way much (e.g. important company correspondence leading to orders almost being lost because they landed in our GMail spam, so I had to turn off the spam filter entirely).
It would be better if we came up with new protocols that solve the email messaging problems and then gradually wean ourselves off email rather than trying to "fix" email.
In a sense this is already happening although instead of migrating to open protocols which solve the email problems we're migrating a lot of messaging to walled gardens which solve the problems (e.g. WhatsApp).
I completely agree with this.
I think techniques like HashCash by Adam Back should be revisited.
Ironic given the amount of crypto spam that exists.
Of course this assumption is fully invalidated by open-access mail service providers: maintaining communication with xXx_kitty_slayer_xXx@tepidmail.com tells absolutely nothing about sincerity and authenticity of messages from DrJohnSmith@tepidmail.com. Not only there's roughly zero correlation, it could be argued that correlation is slightly negative if one does not expect out-of-the-blue contact attempts. Funnily enough, with the critical mass of non-spam migrating to providers, it becomes sort of positive signal, fueling feedback loop: even if the majority of spam came from gmail, messages from gmail would still be more likely to be non-spam than messages from rncrosoft.com, because most non-spam comes from there too.
On top of that, there's a lot of soft-spam. Companies will happily send out critical contract-related information and unwanted-unasked advertising from the very same domain and even the same address. The big providers are not shying away from selling shielding services to spam agencies.
That "someone" could - in theory - require pgpgpg signatures attached to messages and have origin-based filtering/ranking, but good luck convincing people to set up their gmail/outlook to even generate the signature, let alone publish it reliably/revocably. At the receiver end, your only reasonable bet is filtering/ranking the content. It's probably not a strictly uphill battle, but you definitely need significant amount of volume and tagging before your models become good enough and it stops to the case.
In reality, even if it sounds a bit ironic given above, the best route to fixing this is self hosting, especially in the cases where your messages are needed. The more people try to send mail from their self-hosted instances and insisted that it was the problem of the other end to complain to their service providers if they do not receive the messages, the less hegemony the cartel providers would have. And maybe we could have some sort of client auth for email.
And generally I find that the content isn't that high quality. You're getting someone who superficially appears like an expert and looks like they've thought through the problem, but if you know anything about the problem, you'll find that the articles are usually weak (Gell-Mann Amnesia).
* I am still using the same domains that I've always hosted mail for * Since I am sending from new IP addresses, but want to retain the ability to send from the old MTA until it is decomissioned, the new MTA uses a different HELO identifier, and the dependencies for that are all in place (forward and reverse DNS addresses and updated SPF records) * The new DNS records are still in the same administrative domain as the old ones
It's kind of sad. I compare it to when Memes started becoming popular as communication tools on the internet (mid 2000s from what i recall): Instead of writing text to convey the idea they wanted, people just pasted a meme (my face when ... whatever).
Now with AI, everyone write everything in the same way.
I had hoped that something useful would come out of blockchain technology that would allow for a better messaging protocol that was spam resistant, e2e encrypted and decentralized by design but I didnt see anything.
Haven't had an issue since I moved to 365. Costs a little more but so much less hassle. I don't like it but I also need people to actually receive my emails.
I'm still refusing to give up the good fight.
When you start digging in you realize that (like most things) there are multiple different definitions. LLMs are very good at putting together a lot of words that are spell correctly with grammatically correct sentences (better than me - though I'm particularly bad at this). However people often want to add other things like "correct" (the facts are right...), or some notion of not using too many words. LLMs tend to not be as good at those.
That only applies when the article's author just prompted an LLM (and did not supply the LLM with any data you did not have) and then posted the results.
I'd expect most article authors who use LLMs are more involved then that, checking the results and correcting (or telling the LLM to correct) thing that it got wrong.
If the article is from someone who knows more about the subject then you do then that will be more valuable that what you would get asking the LLM yourself (even if the prompt used in the article didn't include anything that your LLM didn't know).
I think if this post had not gone the AI-generated-text-route, some people here would easily have considered it interesting for its content, even if it was entirely AI generated.
FWIW, in my case, I am running the server on Hetzner, even though is supposed to have a bad reputation, but I have been using its IP for more than 10 years for other things (web server, personal services etc) so I guess that helps
Personally, I usually check the comments before deciding to read something so I appreciate that others have saved me the trouble of reading this article in particular. If the author doesn't care enough to use human time to write it then I am not going to use human time to read it.
Presumptuous.
Likening this to restaurant reviews and publishing houses is disingenuous. One is nothing like this and the other is not helpful in that it’s such a generalization of influence as to be not useful. Everything is an influence doesn’t mean anything.
