Prompt injection allows third-party to inject a system prompt with simple message, by inserting special HTML-like sequence (details below). Some of the issues are well-known and pretty old (almost 2 years for Transformers library)

Issues:

• Ollama: https://github.com/ollama/ollama/issues/15931

• HuggingFace: https://github.com/huggingface/transformers/issues/29279 (labeled as feature request) https://github.com/huggingface/transformers/issues/47822 (with a bug label)

• Gemma4: https://github.com/google-deepmind/gemma/issues/768

The problem is that for tools like Ollama there is no solution except of to fix it by Ollama developers

More context

Why this is important: prompt injections are pretty dangerous and as long as user input is an instruction to the model it could be decided as code injections vulnerability. And in combination with long-memory and multi-agentic runtimes this vulnerability could stay in system for a long time. And it has not been decided as a serious security vulnerability by the global community yet

Temporal Solution

So if you're running local models just make sure to throw an error when there is a special sequence in user input. For Gemma family it is <|turn> and for tiktoken-based models it's <|im_start|>. And would be nice to see more solutions

Disclaimer

I'm not a security expert

The companies were notified 30 days ago about the issue. Only Google responded with a feedback on the issue (swiftly)