Jabber/XMPP: 25 Years of Digital Independence(gultsch.de) |
Jabber/XMPP: 25 Years of Digital Independence(gultsch.de) |
(I once found myself in the same meeting as two of the people who made the mistakes that meant you couldn’t just pipe stuff through an XML parser and hope it worked. I tend to speak my mind)
[0]: https://movim.eu/
Of course, that doesn't automatically make it great at what it's trying to do, nor does it make XMPP terrible in practice, but between mixed extension support at the client and server side, with XMPP you can never really know if you're getting "plain old Jabber" or something more like Matrix in terms of user experience.
Debatable, this mostly boils down to the client design choices and priorities, and has little to nothing to do with the protocol. Someone came-up with a metaphor I like to illustrate it: the banquet and the barbecue¹. XMPP has "barbecue"-style clients like Conversations, Kaidan, Dino?, and other that are rather "banquet"-style like Gajim, Fluux, Movim, …
I personally use Conversations on the go and Gajim otherwise. I also happen to have interfaced all my "high-volume/many-participants" IRC chans from back in the days with the biboumi² gateway, so they appear to my XMPP clients as native XMPP groupchats, and the experience was great-enough for me to ultimately drop weechat a decade or so ago and have all my "banquet"-style chans under one XMPP roof.
> between mixed extension support at the client and server side, with XMPP you can never really know if you're getting "plain old Jabber" or something more like Matrix in terms of user experience.
From over a decade of using XMPP daily, this concern is more of philosophical nature than anything. The vast majority of the people you'll reach over XMPP use a decently modern and maintained client that will "just work". Worst case, they will still be getting your messages and the meaning across, because the "message passing" core of XMPP was defined 25-odds years ago and hasn't changed.
The real "risk" for XMPP would be to have a large number of users stuck on an unmaintained client, and users staying behind for years while the ecosystem moves on. We had a bit of that with pidgin 8-or-so years ago. The worse that happens is that you can't use the latest E2EE scheme with them, or that attachments are slower to arrive, this sort of things. That's IMO not bad for a 25 years old protocol that's truly decentralized.
¹: https://blogs.gnome.org/tbernard/2018/05/16/banquets-and-bar...
now Jabber has client issues, but stuff like Monal "basically work" for text + images + chat rooms. I'm excited for Fluux. You can of course set up one of the random web clients. Stuff like reactions "work", notifications is a solved problem.
But like... at a basic level, "I'm hosting a Jabber server for my small community" has a lot less headaches than with Matrix. Yes if your server goes down then you lose the chat. Oh no. Plenty survived off of IRC IMO.
For the HN crowd... I would say that there's still fun jank, but who doesn't like jankiness in their programs for fun? People should swarm to this opportunity to have some fun with busted tooling again!
It's sort of the Apache of chat protocols and that's not a compliment.
I run my own server for 2 people. Using Prosody [0] took some config but wasn’t that hard. Easier than installing Arch, harder than installing Windows ;)
Nowadays, there is also Snikket [1] by the Prosody maintainer, which is essentially an opinionated all-in-one configuration setup based on Prosody, barely any config needed.
[0]: https://prosody.im/
[1]: https://snikket.org/
Just like with Apache, all you need is a sane default configuration that covers the most common use cases, and I think modern XMPP servers offer that.
Comparing XMPP at 11 years old to Matrix today makes it abundantly clear that the main thing Element LLC does is waste European public grant funds
If you allow for independent implementations of the protocol this will always happen. Also, inventing another non-standard protocol creates even more incompatibility between clients and servers.
Working with XMPP gave me a whole new appreciation for why people keep inventing their own chat protocols. I don't think this is an unsalvageable situation, but the protocol is desperately in need of an "XMPP: the Good Parts" doc.
And here I am, hosting a Matrix server for my family for 6 years now, with no issues and an amazing experience, from the backend to the huge variety of mobile clients, a mature Rust sdk that I can hook bots into with ease, etc.
This protocol is XMPP in 99% of the time, both Apple and Google use XMPP to connect the phones to their push notification service.
On Android, some Matrix clients offer the option to use non-Google push notification channel via UnifiedPush. UnifiedPush needs a distributor, a service that runs on the phone and manages the delivery of push notifications. Conversations, a popular Android XMPP messenger, can be used as such distributor.
So if you want digital independence with Matrix and thus do not want to use Apple/Google push, you might end up installing an XMPP messenger, so that your Matrix messenger actually can show notifications. Matrix alone just doesn't provide what is needed for digital independence.
This has worked well since I can spin up new accounts on demand and I can use existing server software (ejabberd, perfect) and clients (Fluux, great; Conversations, okay). I've had to make a few custom modifications to each client to make agent comms a bit nicer (e.g. exposing status messages more prominently since I use that to expose current agent commands), but the XMPP layer has had pretty much everything I could have wanted.
Controlling a chat service is a bit intoxicating, I'm now using it as my notification service over a bunch of different projects.
I used Google Chat way back, its nice that the tech is still around and great as ever.
The benefit beyond that was having agents running on my server directly so they can do a lot of sysadmin and monitoring tasks, as well as suggesting config changes through PRs to my nix config.
pi-msg doesn't support bonjour, I'm running it on a nixos server (nix is the real secret ingredient of my whole setup) the xmpp server is the same machine as the agents. I'm almost always on a different network to the one where the server is. If the server goes down the agents and ejabberd are coming down simultaneously.
I stumble upon Matrix from time to time, but Jabber I haven't really seen anywhere in the past decade after my internet friends moved to IRC. Are there any bigger communities still using it? Back in the days Facebook Messenger was backed by it, so maybe there's still some bigger entity using it as a backbone for something?
Is there anywhere to read more about this?
I admit I didn't research "XMPP-based discord alternative" but maybe that says something about the ecosystem's health.
It's great to see that Openfire (Ignite Realtime's XMPP server) has had a release even last week! https://www.igniterealtime.org/projects/openfire/
If you're coming from Slack, this plugin is a game changer:
https://modules.prosody.im/mod_slack_webhooks
sudo prosodyctl install --server=https://modules.prosody.im/rocks/ mod_slack_webhooks
There was a talk at FOSDEM about that.
I'm all-in on Matrix right now and use it for my family but it's a far cry from the usability that XMPP had. I am still hopeful that Matrix can live up to its promises but perhaps XMPP will always be around if that doesn't work out. Who knows!
1. Gain/find-someone-with crypto library use knowledge.
2. Jabber client with Iroh networking and E2EE.
3. Completely remove XMPP server dependence.
4. Allow users to exchange Iroh-Node-ID outside of jabber (QR code, other secure channels?), store in Jabber contacts.
5. Develop from there. Calendar usage, multiparty, backups.
But I have neither the time nor the knowhow right now.
Finding/viewing/sharing/joining chats
Adding users (can't find them, invites not being received, etc)
Broken messaging (messages not arriving, formatting not being handled or handled the same way, even on the same XMPP server!)
OMEMO issues (key exchange not working, or OMEMO keys getting lost or something when both clients involved were working perfectly fine the day before - Telegram's E2E doesn't break randomly and no, there isn't a MitM issue)
Clients asking for encryption, but then get mad because you're using Thunderbird's XMPP implementation and complain they don't like the encryption supported by Thunderbird.
Not being able to disable encryption on some clients (I actually rather not have encryption anymore because of all the problems)
Multi-device messaging issues (fresh chats can have messages going to one device but not the one the user is active on or history synchronisation)
Mobile notifications inconsistently working, even when using external push messaging providers (it works consistently always on Telegram, it's even more consistent with Matrix, even though Matrix has more push disadvantages)
Client crashes (I can't even remember when Telegram last crashed, but Pidgin crashed on me today and it has only a single XMPP account on it)
Clients failing to connect without useful information (Siskin no longer connects to my account and the only indication is that the "enabled" button becomes disabled, despite the account working on other clients and devices)
UI lockups (Miranda NG, Psi, Gajim, and Pidgin all tend to become unresponsive for short periods of time, probably a blocking wait being imposed in the UI for some reason?)
Common issues that every modern messaging application has, like voice and video, are barely supported overall despite being fully supported in the XMPP standards (file sending is even an issue for Thunderbird)
There is next to no failure recovery on most XMPP clients. Anything that fails on Telegram will try to recover and resend without needing the user to intervene and helpfully displays it's working on it.
All of them seem to look like they're from the 00s!
Except no.
You can't even have a conversation between Conversations and Fluux because OMEMO is enabled by default in Conversations and there is just no OMEMO support in Fluux - despite it being actually modern one.
Gajim works but Pidgin doesn't even show "I sent you an OMEMO encrypted message but your client doesn’t seem to support that". Just like images, which is a first party in Telegram and even WhatsApp.
So much for 'just works'.
My limited experience is that it's not the case. When I tried to join a few rooms I got insulted because my client apparently used extensions that weren't supported by some XMPP enthusiasts' clients, so my client sucked and I had to change it and use an ugly one instead.
I'd rather get a feature-complete protocol than an extensible one, so that at least everyone can speak the same
Not my experience. I had to give up on it since it was just completely flooded with spam, and unlike mail readers that have ways to mitigate it, XMPP clients were ill prepared. Might still work if you keep your address hidden, but as chat-like alternative to a public email it just stopped working years ago.
No. It doesn't "just work". It very much depends on which subset of the XEPs all the parties in the chain support.
> All other popular systems and servers went defunct;
Well, in this sense it does just work :)
It exists! https://docs.modernxmpp.org/
I don't find the UX to be clunky either, but I acknowledge that could just be my familiarity with the program, so I would leave that for others to dispute.
For me, the bigger problem is that I was unable to get OMEMO + Apple Clients working properly. I think OMEMO 1 has some design flaws and hope that OMEMO 2 will fix those.
And I moved to Conversations because I needed a client that wasn't as crashy as Xabber was for me.
Evgeny is a venture capitalist (ew) but the product is solid and they've published two TrailOfBits audits with a third completed and awaiting publication by TOB
(aside long polling which I cannot comment) why is this a even an issue?
Centralized push services (may it be FCM or unified push) are at this point well established.
As a user using a specific messenger app as a push service doesn't gain me anything, it only makes it more difficult to switch messenger app if i choose to use it and mostly use an additional push service (hurting battery life), because most receive push notifications via Google anyway.
I think the right solution is to use a different notification service, not controlled by Google or Apple, not to get rid of the concept altogether.
XMPP uses a persistent connection. These are kept alive using TCP keepalive (both from server and on the client, but client side will be handled by the MPU and thus doesn't need CPU wake-up) and client initiated application protocol pings at intervals of 5 minutes or even less. Client initiated pings are much better on the application protocol, because they can be synced with other operating system tasks and thus don't need any active wake-up.
https://www.sigidwiki.com/wiki/XMPP_trials says "On shortwave, you can see the military use this protocol. They are known for using MIL 188-110A Serial HF waveform (fixed 600bps/S) and 6-bit code clear text with dual bursts of STANAG 4539 and STANAG-5066 as for XMPP Multi-User Chat (MUC) messages, over a bandwidth of 34 kHz. Multi-User Chat (MUC) is a central service for military communication. [..] XMPP is widely used for military deployments, where operation over constrained and degraded networks is often essential, particularly for tactical operation"
It seems their "JChat" application is based on XMPP.
Anyway, there is a discovery mechanism, and clients do adapt the version used by peers, so there is no incompatibility. Kaidan choose for a while to have only OMEMO:2, because resources are limited and they were going for the latest version first, but resource are limited for everybody, and it took times for others to implement OMEMO:2 too.
NLnet/NGI did (and still does) gave financial support on several XMPP projects (including mine, huge thanks to them), and, curiously, when there is money to help, implementations happen. That's problem nb 1 with XMPP: lack of financial resource.
() GUI (notably web UI) is being reworked, not stable yet, but the CLI is pretty solid.
I worked on adding OMEMO2 support to Converse.js via libomemo.js and I tested it against Kaidan.
That's a nice way of putting it and it definitely resonates with my experience.
But why wouldn't it be possible to have these two in the same app as long as they are very cleanly separated in terms of who gets to notify me when, what's displayed where etc.?
WhatsApp (with channels) and Telegram (with its huge groups) seem to address both just fine, as far as I can tell, although I barely use their "mass-messaging/social-media-like" features.
As to whether one protocol can address both use cases: Maybe it doesn't need to either? I could see 1:1 chats and "ephemeral/barbecue" groups using one type of history persistence policy (i.e. usually none) and large group chats/pub-sub-like feeds another. This could all be independent of what holds somebody's persistent identity/identities.
I don't believe it's "impossible" to have both in a same app, it's just that the kind of high-density UI and advanced features you depend upon for high-volume chats rarely intersect with family-scale/1-to-one chats, so you either make compromises that directly affect usability/efficiency, or you essentially end-up with 2 chat paradigms at odds in a same client.
> As to whether one protocol can address both use cases: Maybe it doesn't need to either?
It depends what you mean by protocol in this case. Chat rooms (MUCs in the XMPP verbiage) are handled by a specific component, so it's practically happening the way you describe, but at its core, the "message-passing" (stanza-based) nature of the XMPP protocol doesn't change.
I think it does.
How would you as a client choose that the (potentially) encrypted chat history of all rooms is immediately available after logging in, let alone after joining an encrypted room?
You can't:
- how long messages are stored server side depends on someone else's server
- encrypted messages aren't recoverable from new sessions
8 years ago? Pidgin still doesn't do MAM today.
source? wouldn't that be a license violation?
Don't all actively maintained servers support persistent message archive management now?
The only downsides I've experienced are:
1. Images and accompanying text get separated, which turn-based bots get confused by. I seem to recall there's a standard for this but it's not implemented.
2. XMPP support isn't common in claw-type agents so I've had to patch it in. Not a terrible experience overall.
I had a look at Matrix but bounced off Element, hard.
I found this pretty baffling/sad. Why do these agent harnesses have messaging plug-ins for the most obscure messaging platforms but not XMPP? It seems like a no-brainer!
People would apparently rather pay for a second number/SIM/phone dedicated to Openclaw on WhatsApp and suffer periodic outages due to Meta actively discouraging automation use than set up a minimal XMPP server and client...
Even setting up Matrix (at least with E2E encryption) was incredibly painful. It worked in the end for me, but the amount of tokens I've had to burn... And I don't even want to think about the next update.
Eh, last time I tried I still got leftover notifications for messages I had long read and even replied to on a different client hanging around on my other devices. (To be fair, Matrix does the same thing too, but only in roughly 50% of all cases, which is much more infuriating than something I can at least understand as a fundamental protocol limitation and not a weird implementation bug.)
I've been using XMPP on and off over the past 20 years and I would really love it to succeed, but papercuts like these need to be solved before I can recommend it even to technically open-minded friends.
> But like... at a basic level, "I'm hosting a Jabber server for my small community" has a lot less headaches than with Matrix. Yes if your server goes down then you lose the chat. Oh no. Plenty survived off of IRC IMO.
It might be great for that, but by the same token it's quite terrible as a persistent personal messaging handle. Durable identity/handles shouldn't be so bound to a community, since it makes it very hard to switch and you need to put an incredible amount of trust in the admin(s) as a result. (This is a tangent to the Matrix vs. XMPP point, as both have the same issue, as far as I can tell.)
Ideally, I'd like to have me@mydomain.com as my persistent XMPP ID that I can freely migrate across servers just by updating a DNS record without having to worry too much about who is actually hosting the server. The AT protocol does this right.
I suppose here you're saying "I should be able to do this without providers having to do work"? Like you show up with lxgr@foo.com to bar.com and create an account, and everything kinda fits into place (modulo you having to set up some DNS?)
It is already possible of course to have XMPP server at domain A handle things for domain B as well. But it is cool how ATProto seems to really bake in the concept of "all domains are welcome".
I might lose my chat history and maybe my contact list if I don't have a local backup of it, but people would still be able to reach me as lxgr@example.com. Ideally, I'd be able to upload my local, end-to-end encrypted backup to xmpp.bar.com and continue exactly where I left off.
AT proto's main innovation to me is decoupling the concerns of user identifier and data host at the protocol level, in the same way that DNS and my web or mail host are decoupled.
What server and client(s)? I have not seen this issue with ejabberd + Conversations. Might also be a matter of enabling the right extensions on the server.
Servers: https://matrix.org/ecosystem/servers/
Clients: https://matrix.org/ecosystem/clients/
Also, imho Matrix is a better standard than xmpp, because of what PunchHamster said and the better feature set. Of course, everyone is entitled to their own opinion :)
Id rather someone be able to reach out to me with a note the chat is insecure, rather than be unreachable at all
not it isn't - still today there's no choice between incompatible room types or incompatible encryption algorithms and whatever else.
> And, apparently from your comment, there's only ever going to be one encryption algorithm, because cryptography is a solved problem now?..
That's not from my comment. If there will be another one it likely will be integrated into THE specification such that it will NOT divide the ecosystem into "I prefer MEGOLM" and "I prefer to implement new thing".
AFAIK, even today you can fetch the WhatsApp Android apk, extract the zip and do a `strings classes*.dex | grep signal` to find class and package names matching the sources in the signal-protocol-java repository in the Signal GitHub.
Edit: now I can't stop thinking about correlation vs causation - you being under the boulder bleeding out at the same time XMPP encryption suddenly stops working.
Physical hardware certificate thinggy which got crushed?
(The premise in my understanding is that you own example.com and control its DNS records right? And not "well I hosted on foo.com as name@foo.com but it went down and now I want to move")
[0]: https://forgejo.ellis.link/continuwuation/continuwuity/issue...
- you would be at the mercy of very real and frequent federation problems (Matrix being so implementation-dependent and centred around Synapse ; like, the json serialization format is better described as "whatever order and structure python's stdlib is doing"¹)
- there is no non-destructive way back to another server (the open federation would just reject you, and you would have to "start anew"). Interestingly, that last one pushed some servers over the fence and into XMPP, and they could then switch implementations (e.g. from prosody to ejabberd) effortlessly.
¹: https://telegra.ph/why-not-matrix-08-07#:~:text=matrix%20hom...
2) Why would open federation reject you? Given you don't throw away your private key
2) What do you even mean? I've literally deleted my entire database and then spun up a new server on the same domain before, it does not matter, other servers do not care! The worst that could happen is messages from the original server can no longer be verified as valid - and even that isn't a problem in practice.