Show HN: VLLM with JSON Guided Generation(outlines-dev.github.io) Our project, Outlines, now offers guided/constrained generation (e.g. according to a JSON schema) via the VLLM library. My colleague, Rémi, created some patches that allow one to pass vLLM a JSON schema along with the prompt, which dramatically simplifies deployment of JSON-guided generation. He also added a new `serve` interface that puts it all together and makes serving such models a 2-3 line process. Check it out and tell us what you think! |