Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot(this.weekinsecurity.com) |
Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot(this.weekinsecurity.com) |
I'm not sure "worked properly" and "as intended" accurately describe this situation.
I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.
[1]: https://www.documentcloud.org/documents/28202858-meta-ai-ag-...
But it's important to acknowledge that there was a 'bug' in an underlying tool and not in the chatbot, and still PIP/fire those responsible for publishing the chatbot and exposed an otherwise internal tool to the public, and not those that introduced the 'bug' to an internal tool.
The problem is when the backend function doesn't verify that the email matches the username.
Or perhaps said different: use the submitted info to identify the account; send any sensitive messages (recovery codes, password resets whatever) to only the contact info on file. If the chat bot can send such email it should do so via an API that sends only to contact info on file for the associated account and not to an email that's provided by the bot.
But when humans handled it, this was not as much as a problem. That is, the humans did the job, because they recognized the need to do that job.
Sure sometimes accounts could get recovered if a human was tricked, but evidently it was easier to trick the LLM in masse than humans.
What I gather is that this internal tool was used by human support agents, and it was their responsibility to verify the email adresses and general validity of a claim.
But when implementing AGI TM that was overseen, maybe the oversight in the separate code path was a 'bug', but the mistake was making the chatbot obviously, if the separate code path had a bug, then it had become ossified into a feature, and it was internal, not exposed to the public.
This is an external communication, to save face sure, but if this is the internal excuse, it would be absolutely the wrong RCA and it reads as if the one who made the mistake is not admitting they made their mistake. Which to be honest, just making the mistake is enough to get fired, but not admitting it is enough to get ultra fired.
The compromises allowed the hackers to take over the person's entire Instagram and any linked accounts, including obtaining contact information, dates of birth, and profile information, as well as the ability to access the person's posts, direct messages, and account activity [...]
the hacks began around April 17 and lasted until this week [...]"
This is staggering.
(If anyone at Meta/Instagram sees this I wrote a brief blog post with the details. Please help! https://addisonwebb.com/blog/2026-06-05-Can%20Someone%20at%2... )
If this doesn't work, I'd encourage you to reach out to a brand/ad agency and pay them $100 to ask their meta contact to help you get unblocked. You pretty much have to know someone who knows someone at meta in order to create these.
Tip: Do not post about this on twitter or other platforms - you'll get a ton of automated spam.
[1]: https://rainermuehlhoff.de/KI-und-der-neue-Faschismus-Reclam...
oh no...Meta what are you doing
...They really ahouldn't have, and I wonder how this will affect all the big AI IPOs. After all, Meta is one of the big players in the space. Surely if they can't do it right, then...
That in turn means three things... it costs a lot of money to have humans look at these tickets, the PR damage from both acting and not acting on such requests can be immense, and users/customers can be anything from the smartest and richest people on the world down to the kind of utter imbeciles whose brains get surpassed by bears [1] or who plainly are not able to write. To make it worse, often enough online services don't have any kind of tie back to some known government-issued ID (either directly or by a proxy such as a mobile phone SIM), there's corruption involved on all levels, and for particularly "juicy" targets the stakes, if they can be converted to a monetary amount at all, can reach into the millions of dollars.
Now, Instagram alone has 3 billion (!) users from across the world, so they are bound to not just having to spend a lot of money on user support (remember, we are talking about the entire world, they also need to deal with about 7.000 (!) actively spoken languages, and having attack targets that are as powerful as US Presidents or as rich as Elon Musk. Clearly, the risk management involved in the entire idea was horribly deficient, but let's not act like this is a trivial problem domain in the first place. And hence the push for AI, simply because it - if done correctly - can take a lot of work off of the first-level support desks for a fraction of the money.
[1] https://velvetshark.com/til/til-smartest-bears-dumbest-touri...
[2] https://www.sapiens.org/language/world-languages-counting-me...
During development they were likely not thinking of the user experience, nor even the support agent experience, but on their development experience, they asked the LLM to develop the chatbot, and it worked, and the speed was documented and reported upstream so that shareholders invest, if there is any forethought it would go against the narrative of AI becoming the engineer or 100xing productivity.
https://www.maine.gov/agviewer/content/ag/985235c7-cb95-4be2...
> Date Breach Discovered: 05-31-2026
It's like, people abusing an open door. "Guys, just because we left the door open to your bedroom doesn't mean we're responsible".
God can only hope this is a business ending lawsuit.
also this is more like them leaving the keys in the door, then someone comes along, uses the keys, and steals all your stuff.
truthfully, no equipment is actually defective in this scenario eh?
Meta believes that they can vibe-code their reputation down the drain by removing humans in the loop.
Applying a technical solution to a social problem almost always ends in disasters like this.
Reputation can’t be vibe-coded.
I'm creating the accounts in Meta Business Suite, so I would have a recourse with my main personal account which can be linked to some adspend, so I'm assuming it will have better support channels than accounts created through an end-user interface.
In fact it's arguably a feature. The ability of support staff to short-circuit nitpicky rules when there's an obvious external validation happening (e.g. you're on the phone with a user who's presenting ID in real time and correlating it with previous use of the account, etc...) makes for better data quality and happier customers.
Obviously, yes, you can then human-engineer an authentication breach. But that was very difficult, because people are "common-sense careful" in a way we haven't been able to tease out of AI yet.
In principle, it could be designed to do so to handle cases where a new email address has been confirmed out of band, e.g. for an account representing a company or a political office. But that's a relatively unusual situation, not something you'd want to be available to every user writing in. (Even if you had an all-human support department, this sort of functionality would only be available to a select few agents.)
> If we’re going to talk about good software design, we have to talk about Laziness, Impatience, and Hubris, the basis of good software design.
sourced from https://bcantrill.dtrace.org/2026/04/12/the-peril-of-lazines..., where Bryan Cantrill makes the point that:
> The problem is that LLMs inherently lack the virtue of laziness. Work costs nothing to an LLM. LLMs do not feel a need to optimize for their own (or anyone’s) future time, and will happily dump more and more onto a layercake of garbage.
which I think is interesting, albeit somewhat tangential to the current discussion.
> The LLM correctly generated tokens according to user input, however due to a bug in a separate code path, the system did not properly verify the email address
> Nginx correctly handled the user requests according to the HTTP standard, however due to a bug in a separate code path, the system did not properly verify the email address
Don't know if AI is to blame, but I've used to see these kinds of nonsense post-mortems even in the pre-llm era, and it's always due to some internal fighting ongoing between various departments.
(Usually said jocularly when everyone is at their most upset, e.g. a vacation ruined)
I agree with you that in a week nobody will be talking any more, but I'm pretty sure it's a GDPR data breach, and they can have some trouble within EU.
Yeah, they probably don't give a fu.. about EU, but if the response doesn't matter at all why did they spend time on it?