- Performance. Unreasonably long amounts of time to send out e-mails. phpList degraded to the point of taking several days to process a campaign. listmonk can spawn N goroutines (~threads) and push e-mails to multiple SMTP servers. On a commodity ec2 instance, we're able to send 1.5mn+ e-mails in a couple hours.
- Subscriber imports were extremely slow. Direct integration to keep subscribers in sync with external CRMs was cumbersome. Direct DB inserts were complicated due to the complex table structures. listmonk imports 10k records/sec into a Postgres DB on a commidty ec2 instance.
- Segmentation. Often, we have to rapidly segment users by custom attributes and conditions and relay an update to them. listmonk supports SQL expressions to segment users on their attributes that are defined as arbitrary JSON maps (thanks to Postgres JSONB type).
- Unavailability of dynamic templates. listmonk templates support Go template expressions so it's possible to write logic in messages to make them dynamic.
I would like to add that listmonk is still work in progress and requires a number of essential features before it can come out of alpha.
Are you guys using returnpath' senderscore or a similar certification to send that amount of emails ?
So, is it connected to some kind of CRM ?
About attributes, every subscriber entry in listmonk can take a JSON map of attributes that can be queried.
Not sure what SMTP setup you had with phpList, but sending 1.5m messages on a VPS should take less than a day with the right Postfix tweaks.
We did try several things; Postfix tweaks, SES / Postal SMTPs, rate changes in the phpList config etc., but nothing seemed to work. This, along with the other factors (segmentation, dynamic templating), lead to listmonk.
This looks like the free/open-source version to Sendy based on more modern tech like Golang/Docker/etc I've been hoping for, congrats on your alpha release!
What are best practices for using/selecting an ESP if you were to use a project like this and want to ensure reasonable deliverability?
I've been looking for a good solution for ad-hoc email campaigns (w/ templates) and for sending "systems" emails to an audience of customers.
I currently look after lists on both MailChimp and EmailOctopus and get frustrated by the flat pricing for the size of the list and not how you engage with the list (e.g. I'd love to see: store your list for $X/mo, or pay $X + (list size * $Y) in the months when you email the list).
The issue is that you need to put lots of effort into getting off of blacklists, or your email won't be delivered.
As a business that sends emails infrequently, it's probably saved me hundreds/thousands and it has good instructions to set it up with SES. One-time (ish) fee, until you need to upgrade to the next major version.
Besides, that an email is produced by an AGPL software does not make it covered by the AGPL. That should be exactly the same legal situation as with open source text editors.
So, who does need to be provided with a copy? Just the 'administrators', the people using it to send email?
What about if I modify it and repackage it as monklist or whatever, and charge people to use it. I must give them the source code for listmonk too, and, since another requirement is same licence, the source for my modified version? i.e. monklist couldn't be closed source, but myapp that happens to use listmonk unmodified for sending emails can be?
Have you considered using a hosted platform that uses Amazon SES rather then hosting Sendy or other tools yourself? I own BigMailer and our clients with established SES accounts (high daily rate limits > high limit per sec) see 1m emails go out in a little over 1 hour.
I always wonder why people choose to host email software in house and it seems like the requirement for direct integration with in-house system(s) is one of the biggest considerations when choosing in-house vs. off-the-shelf solution.
listmonk is a replacement for Sendy. You can connect your SES account to it using the SES SMTP interface.
Integration with in-house DBs was a top reason for us.
Try to implement bounce processing soon - an average list decays at 2% per month (3% for B2B lists) so in a few months, the bounce rate can be high enough for mail box providers to start blocking your emails due to attempts to send to large number of invalid email addresses. Especially sensitive issue with your list size and sending volume.
There might be a couple more useful nuggets in this article on email deliverability best practices https://www.bigmailer.io/blog/improve-email-deliverability-b...
I don't see anything in the docs about supplying an unsubscribe link. I don't see an endpoint in the API that could be used with an anchor tag in the footer of the email. Is this something that is supported?
And: my client is concerned about email/spam regulations, which I'm very unfamiliar with. Is there anything I'd need to look out for in this regard?
PS: Postal SMTP is awesome.
starting to sound hopelessly naive
you deliver email updates to fund LPs that aren't unilaterally marking you as spam for no reason
Using an ESP will generally simplify the amount of work necessary to achieve good delivery, but due to the complexities of domain reputation and fingerprinting, very occasionally they can also be the cause of delivery problems.
General guidelines:
- It's a bit harder to predict how deliverability will turn out at smaller ESPs, vs. well-established ones like Sendgrid, SES, Mailgun, and perhaps Mailchimp.
- DKIM, SPF, DMARC (p=none is fine to start; gmail has a guide for deploying dmarc https://support.google.com/a/answer/2466563)
- Use the same root domain everywhere within your email: From, Return-Path, DKIM signature headers; URLs for links and images in the message body. (Separate subdomains are fine, e.g., img.example.com, bounce.example.com)
- Having your sending domain show up in third party email can cause problems. Try to avoid it if possible; use a separate domain as a last resort.
- Send consistently. Some variation in sending patterns will be tolerated, but you'll probably have trouble if you send 500 messages a day most of the time, then once a month you send 100,000 messages.
- Consider a dedicated IP if you're sending over 50,000-ish messages a month.
- You may want to split up transactional email from marketing, sending these from separate IPs and/or subdomains.
- Finally, make sure you're sending email your recipients find valuable and are interested in receiving. Too much disinterest/disengagement can cause delivery issues.
I would recommend to NOT send different email types from different providers, like use SendGrid or SES for transactional emails and then some other platform for marketing emails, especially using the same sender domain. That's because email header signature and tracking links will be different and email box providers like Gmail can find it suspicious and send emails to Spam. And then there is the issue of processing and syncing bounces/unsubscribes/complaints data.
Keeping all email campaign types in a single platform allows for centralized processing of bounces/unsubscribes/complaints date and protects future campaign engagement + sender reputation.
Also, the quality and size of the starting list matters too because a large list (like the OPs) or unengaged list can have a high bounce and low engagement rate, which factors into future deliverability.
Now before you mention how this should be a solveable problem inn AWS, we’re already well underway testing with the client. The problem presented itself as a manifestation of inheriting the last guy’s infrastructure and working to...unfuck a lot of his questionable design choices.
https://returnpath.com/solutions/email-deliverability-optimi...
Maybe look at https://tldrlegal.com/license/gnu-affero-general-public-lice... first. The AGPL GNU page https://www.gnu.org/licenses/agpl-3.0.en.html also covers questions and has an explanation. I think the summary covers it already:
> It requires the operator of a network server to provide the source code of the modified version running there to the users of that server. Therefore, public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version.
So the license requirement is for the users of the software directly. They need to get access to the modified source.
> So, who does need to be provided with a copy? Just the 'administrators', the people using it to send email?
If that is a closed group said group. If it's the public the public. So in your case yes, the administrators.
You could make the argument (it has been tried before with editors) that the email is part of listmonk, but reasonably it would not stand. At first look there is more weight behind the argument that accessing the email is a form of network access between the user and listmonk, vie email, but I don't think that would hold up either, given the implications with regards of what is a software and what is a product.
> What about if I modify it and repackage it as monklist or whatever, and charge people to use it. I must give them the source code for listmonk too, and, since another requirement is same licence, the source for my modified version?
I don't think that you have to give the original source of listmonk, but you have to give users - including those who access the software on a server, like in a SaaS scenario - access to the source code of monklist, yes.
> i.e. monklist couldn't be closed source, but myapp that happens to use listmonk unmodified for sending emails can be?
There it gets murky. There are different understandings of when a software with different parts becomes one thing and thus the AGPL and GPL apply, and afaik that's not very well tested in court. That said, yes, if myapp just uses listmonk and is not a modified version of listmonk and listmonk is not an integral component of it, then myapp is not influenced at all by the AGPL. AGPL talks about "modified versions of software", myapp would be no such thing.
That said, you may be surprised by the number of large organizations that have a dozen or more services that send mail on their behalf, for various different purposes... it's not optimal, from a deliverability perspective, but it's also not usually going to cause deliverability issues by itself as long as everything else is in order.
Just saw an email from Starbucks in the spam folder - all because of a few words that no doubt triggered the spam filter.
This sounds hopelessly naive to me. If the receiver marks it as spam, it's spam. By definition. Work harder to only send emails to people who want them.
but that shouldn't effect the deliverability to everyone else, but thats a reality which has its own problem and has its own countermeasures.
when people asked OP if they had a solution to this, they hand waived it away. all closed sourced mailing list clients have solutions for this.
but anyway since somehow it is 2019 and people want to act like its 2009 on this matter I'll just skip ahead to echo the productive answer someone else left:
this could probably be run through Amazon SES with their DKIM settings
Spam is unsolicited commercial email.
if I sign up on a site, click the 'yes email me' button, and then mark the email as spam when I get it, I am the one acting incorrectly, not the sender who is literally doing what I instructed them to do..
Unsubscribing from email lists can be cumbersome and non-uniform, which is usually why people just hit that spam button instead. It always provides them a uniform interface and largely achieves the same goal from their perspective.
People forget subscribing and hit "Spam", which has an impact on the sender reputation and affects future deliverability for that sender.
One of the factors I didn't mention in previous comment is presence of physical mail address.
Not sure what "unfortunately for me" is in reference to - BigMailer isn't a platform for cold emails (there are plenty others that specialize in cold email campaigns) and Amazon SES doesn't give a free pass on the practice either.