New Mixtral HQQ Quantzied 4-bit/2-bit configuration(huggingface.co) |
New Mixtral HQQ Quantzied 4-bit/2-bit configuration(huggingface.co) |
Base: https://huggingface.co/mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-a...
Instruct: https://huggingface.co/mobiuslabsgmbh/Mixtral-8x7B-Instruct-...
Shout-out to Artem Eliseev and Denis Mazur for suggesting this idea ( https://github.com/mobiusml/hqq/issues/2 )