KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44x fewer params. Drop-in llama.cpp integration.

TurboQuant for GGML: 4.57x KV Cache Compression with 72K+ Context for Llama-3.3-70B on Consumer GPUs.

No description available

TurboQuant KV Cache Compression for llama.cpp — 5.2x memory reduction with near-lossless quality | Implementation of Google DeepMind's TurboQuant (ICLR 2026)

No description available

1bit llama.cpp gguf weights paired with turboquant 4 bit kv cache

Complete local AI coding pipeline: Qwen3.5-35B-A3B + llama-server + TurboQuant + OpenCode + Context7 MCP + Chrome DevTools. 188 t/s on RTX 5090, zero cloud APIs.

No description available

Fused Triton kernels for TurboQuant KV cache compression — 2-4 bit quantization with RHT rotation. Drop-in HuggingFace & vLLM integration. Up to 4.9x KV cache compression for Llama, Qwen, Mistral, and more.

Native Windows build of vLLM v0.17.1 with Triton support and TurboQuant KV cache compression — Qwen 3.5, Llama 4, and more. No WSL, no Docker. Pre-built wheel + patchset for MSVC 2022 + CUDA 12.6.

Experimental TurboQuant implementation and llama.cpp-style integration path for long-context inference

World's first TurboQuant KV cache compression for llama.cpp on AMD ROCm (RX 9070 / gfx1201)

Practical guide: TurboQuant KV-cache quantization on consumer hardware (RTX 3090) — 100K context, 4.3× compression, ICLR 2026

🚀 Run any LLM on any hardware. 130% faster MoE inference with ExpertFlow + TurboQuant KV compression. Ollama-compatible API. Built on llama.cpp.

Simple all in one build script for llama-cpp-turboquant on Windows 11.

Model Switcher & Benchmark Tool for llama-server with TurboQuant KV-Cache

llama.cpp fork: Qwen 3.5 hybrid GGUF + loader fixes; syncs with ggml-org/llama.cpp

Run Large Language Models on CPU with up to 8× less RAM using advanced KV cache compression.

TurboQuant llama.cpp fork with optimized turbo4 kernels for Gemma 4 D=256/512 heads — lazy K/V, batch decode, warp-cooperative write. 120 t/s with 3.8x KV compression on RTX 3090.

A TurboQuant implementation with Llama.cpp for AMD with Vulkan runtime

A lightweight desktop application for managing and interacting with llama.cpp models through a clean, modern interface - Support TurboQuant technology

llama.cpp-turboquant

No description available

TurboQuant + KDA (Kimi Delta Attention) fork of llama.cpp — novel state matrix quantization for linear attention models

Improved TurboQuant quantization for llama.cpp — adding QJL residual, residual window, asymmetric K/V to turbo-tan fork

No description available

No description available

llama-server with TurboQuant (TQ3_0) KV cache compression — DGX Spark ARM64 build

No description available

llama.cpp + TurboQuant CUDA; syncs with ggml-org/llama.cpp