I made a tool that helps alert teams when model outputs change or produce unexpected outputs. My tool creates a recurring scheduled job that hits your endpoint and runs a list of straightforward filters to keep your models in check. The idea is to provide a line of first defense and auditability when working with LLMs/Text-Generation at scale.

Website Link: https://sntnllm.com

Docs Link: https://sntnllm.com/docs

Thanks!