We've been working with Docker to make running quantized models easier with Docker. This integration allows users to pull and run Unsloth-optimized models using the native Docker CLI. Key features: Uses docker model run syntax. Supports Dynamic GGUFs (handling quantization overhead). Cross-platform support. Example usage: docker model run ai/gpt-oss:20B Happy to answer questions about the implementation or the quantization methods used. |