This is devastating. Amazing job Riley for discovering this and thank you for being a defender of the public by warning people about this huge issue!!

I must also say that I'm saddened that OpenAI really hasn't learned any sense of caution from their poor handling of the similarly gaping security hole that is indirect prompt injection.

The image-based hidden prompt injection Riley shows here could be combined with image-based ChatGPT data exfiltration, since a user asking about images might not be surprised to see an image rendered by the chatbot in the context of "perhaps you might be interested in similar images like this."

https://systemweakness.com/new-prompt-injection-attack-on-ch...

Incredible work once again Riley!!

-- UpwardBound, part of the team at Preamble which first discovered direct prompt injection. (https://www.preamble.com/prompt-injection-a-critical-vulnera...)

By itself, it seems like a rather funny party trick, but the applications may be more serious as shown in one of his self replies where a resume can silently influence GPT to rate the applicant highly.