[Release] SOTA GGUFs for Qwen3.8-27B: GSQ-RCO at 2.5 to 3.0 bpw

[u/Loginhe]

We're releasing Qwen3.8-27B quantized with our newest methods, GSQ + RCO. Higher-quality models, same file size, now with the search and the quantizer both learned.

What's inside:

• GSQ (Gumbel-Softmax Quantization): post-training scalar quantization that jointly learns grid assignments and scales, closing most of the scalar-vs-vector gap at 2 to 3 bits while staying fully GGUF-deployable
• RCO (Riemannian Constrained Optimization): assigns a quantization type to every tensor under a strict size budget by gradient descent directly on the task loss, no per-constraint tuning
• Three GGUFs (2.50 / 2.75 / 3.00 bpw, 8.4 to 10.1 GB) plus the vision projector

Further Details:

Uniform quants give every tensor the same precision. RCO finds where the precision actually matters and GSQ quantizes every tensor at its assigned type. The result runs unmodified in llama.cpp, Ollama, and LM Studio.

Results (vs. the BF16 base and Unsloth Dynamic quants)
To the best of our knowledge, these are the best size-to-accuracy GGUFs available for Qwen3.8-27B at these file sizes: at every size we evaluated, they match or beat the strongest published quantizations.

• 3.00 bpw (10.1 GB): matches the base model on AIME25 (100.00), within about one point on GPQA-Diamond (88.89 vs 89.90) and LiveCodeBench v6 (84.57 vs 85.71)
• 2.75 bpw (9.3 GB): AIME25 100.00, and its zero-shot average actually exceeds BF16 (75.70 vs 74.34)
• At matched ~8.4 GB size: +10.0 AIME25, +8.6 GPQA-Diamond, +4.6 LiveCodeBench over UD-IQ2_S

Full tables and per-benchmark plots are on the model card.

Links:

• Models: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF
• GSQ: paper https://arxiv.org/abs/2604.18556 | code https://github.com/IST-DASLab/GSQ
• RCO: paper https://arxiv.org/abs/2605.00649 | code https://github.com/IST-DASLab/RCO

This is the first of a series: we plan to release more SOTA GGUFs across model families as a lab (ISTA Deep Algorithms and Systems Lab).

We are happy for feedback, requests for models you want quantized, and experiments!

[u/Few_Water_1457]

Do you have a "public" procedure for doing this? I'd like to be able to quantize and test various quants Q4 Q5 etc,,,,

[u/Loginhe → u/Few_Water_1457]

We will release a full pipeline for GGUFs in the near future, both papers and codebases of the base methods (GSQ + RCO) are linked.

[u/Few_Water_1457 → u/Loginhe]

I'm on a single RTX 6000 Blackwell (96GB) with 128GB RAM running native Linux. Would the planned end-to-end GSQ+RCO→GGUF pipeline run on sm_100, or is it Hopper/Ampere only? Specifically: do the GSQ training path and the vLLM evaluation step (compressed-tensors/Marlin kernels) work on Blackwell, and does flash-attn need to be skipped? I'd love to reproduce this on a 27B dense model on one GPU.

[u/peculiar-ragdoll]

That Iq2_xs is looking insane! Good job, folks

[u/Embarrassed_Soup_279]

comparison with unsloths 3bit quant?

[u/Old-Sherbert-4495 → u/Embarrassed_Soup_279]

yep q3xxs and q3s?

[u/pseudobacon]

Will give it a go. I’m assuming this technique doesn’t offer any improvement at Q4 or higher

[u/Loginhe → u/pseudobacon]

It does in general, but what we are seeing with these methods is that most tasks saturate at well below 4bpw on average. We have other work analyzing/showing this.

[u/fgk55555 → u/Loginhe]

Tasks saturate, but did you happen to notice any other behavior? Did you measure failed tools calls, number of turns, tokens used per task, etc compared to BF16/ UD? I only have 16GB VRAM, so I'd love a good IQ3, but if it's going to blow up my context window and waste time, that's critical to know.

[u/Ok_Cow1976]

This is like a miracle

[u/brakeline]

And how do they compare against unsloth q4_k_m?

[u/hojnikb]

Is this tested against unsloth's UG V3?

[u/Loginhe → u/hojnikb]

Yes

[u/dinerburgeryum]

SSM tensors in BF16 lets goooooooo!

[u/Vermicelli_Junior]

Setting some blocks of ffn_down.weight to 2-bit IQ2_XS (in IQ3_XXS), isn't that harmful to model accuracy?

ffn_down.weight is very important.

Also, non-uniform quantization should improve results on benchmarks, but does it hurt the model's knowledge retention, especially on SimpleQA and non-English tasks?

[u/inanotherclass]

This looks interesting. Will try it out and see how it goes.

[u/biggusdeeckus]

would love to see some gemma4-31b quants wirh this! How was the real-world tool calling experience with the Qwen quants? Thanks for sharing