Show HN: I made a library for LLM prompt injection/exploit/jailbreak detection

Show HN: I made a library for LLM prompt injection/exploit/jailbreak detection(github.com)

11 points by last_layer 2 years ago | 19 comments

ukuina 2 years ago |

This makes it difficult to justify a production deployment:

> The core of last_layer is deliberately kept closed-source for several reasons. Foremost among these is the concern over reverse engineering. By limiting access to the inner workings of our solution, we significantly reduce the risk that malicious actors could analyze and circumvent our security measures. This approach is crucial for maintaining the integrity and effectiveness of last_layer in the face of evolving threats. Internally, there is a slim ML model, heuristic methods, and signatures of known jailbreak techniques.

beardedwizard 2 years ago | |

So security by obscurity, to defend llms that are routinely exploited from a position of obscurity. This does not inspire confidence. I'm eagerly awaiting second wave of solutions to this problem that don't take a web app firewall approach where context about what is being defended is absent.

simonw 2 years ago | |

Yeah, I don't like this at all. If I'm going to evaluate a prompt injection protection strategy I need to be able to see how it works.

Otherwise I'm left evaluating it through wasting my time playing whac-a-mole with it, which won't give me the confidence I need because I can't be sure an attacker won't guess a strategy that I didn't think of myself.

This doesn't even include details of the evals they are using! It's impossible to evaluate whether what they've built is effective or not.

I'm also not keen on running a compiled .so file released by a group with no information on even who the authors are.

kylebenzle 2 years ago |

I DO NOT like this. I do not like this very much. Why work so hard to make a technology less useful.

Like a way to block certain google searches you don't agree with.

simonw 2 years ago | |

You may be confusing prompt injection with jailbreaking. You should care about prompt injection even if you don't care about jailbreaking:

https://simonwillison.net/2024/Mar/5/prompt-injection-jailbr...

    Prompt injection is a security
    issue. It’s about preventing
    attackers from emailing you and
    tricking your personal digital
    assistant into sending them your
    password reset emails.

    No matter how you feel about “safety
    filters” on models, if you ever want
    a trustworthy digital assistant you
    should care about finding robust
    solutions for prompt injection.

To be fair, this library attempts to solve both at once.

Zambyte 2 years ago | | |

You can avoid prompt injection by simply not using LLMs as autonomous agents where the output of the model is critical for security. That sounds like a horrible idea anyways. A language model is the wrong interface between untrusted people and sensitive data

ravroid 2 years ago | |

You're equating this to censorship, when I think it's more like Google adding security measures so you can't break their search engine rather than removing unfavorable results.

smokeydoe 2 years ago | |

How is it less useful to have more protection from abuse on your LLM service? This is for people implementing their own public products right?

Conasg 2 years ago | |

There's nothing wrong with blocking prompt injection for a customer service chatbot, though. This would be obnoxious applied directly to something like ChatGPT, or worse yet their API, but I don't think that's really the intended use case.

CuriouslyC 2 years ago | |

The technology was always the same amount of useful. This library just lowers the competence floor for exploits. Try to exploit yourself and come up with anti-jailbreak prompts to mitigate the exploits.

Prompt injection is a security issue. It’s about preventing attackers from emailing you and tricking your personal digital assistant into sending them your password reset emails. No matter how you feel about “safety filters” on models, if you ever want a trustworthy digital assistant you should care about finding robust solutions for prompt injection.