⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, + iOS iPhone app.

TurboMLX v0.1 Research Preview public source tree for Qwen3.5-focused MLX TurboQuant experiments.

TurboQuant KV cache compression for MLX with fused Metal kernels. 4.6x compression at 98% FP16 speed.

Extreme KV Cache Compression (1-3 bit) for LLMs natively on Apple Silicon (MLX). Features TurboQuant, asymmetric PolarQuant caching, and OpenAI server compatibility.

A proof of concept of googles TurboQuant Paper https://arxiv.org/abs/2504.19874

First MLX implementation of TurboQuant KV cache compression for Apple Silicon

Vector compression with TurboQuant codecs for embeddings, retrieval, and KV-cache. 10x compression, pure NumPy core — optional GPU acceleration via PyTorch (CUDA/MPS) or MLX (Metal).

MLX benchmark: Gemma 4 + Qwen 3.5 on Apple Silicon with TurboQuant KV cache

TurboQuant KV cache compression for MLX (Apple Silicon)

TurboQuant (ICLR 2026) ported to Apple Silicon — KV cache compression with MLX Metal kernels + PyTorch CPU

First MLX / Apple Silicon native implementation of QJL and TurboQuant

TurboQuant KV cache compression for MLX-LM — run longer contexts on Apple Silicon with 5x less memory

Based off of https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/ and its implementation via MLX

No description available

Experimental TurboQuant-inspired KV-cache backend for MLX Qwen3.5 with baseline and MLX KV quantization benchmarks

Simple and fast inference server implementation for Prism ML 8B MLX 1bit

PrsimLabs Bonsai 8 bit model on MLX using turboquant, tested on M2 pro giving phenomenal tps

[ARCHIVED] Local AI agent framework for Apple Silicon. Superseded by Claude Code and Ollama MLX. See eden-models and mlx-turboquant for active work.

A from-scratch MLX implementation of TurboQuant for near-optimal, 2.5-bit LLM KV cache compression on Apple Silicon.

A lightweight, professional chat UI for running mlx-community vision-language models on Apple Silicon via mlx_vlm. Generates tokens significantly faster than Ollama or LM Studio on the same hardware, with a longer context window and support for modern compression techniques like TurboQuant.

TurboQuant KV cache compression for local LLM inference — 80% memory savings, near-zero quality loss on 8B+ models. PyTorch + MLX (Apple Silicon). Based on arXiv:2504.19874 (Google Research, ICLR 2026).

Turboquant mlx implementation experiment

MLX proof of concept for TurboQuant-style KV-cache compression, with dense vs quantized inference comparison, batch experiments, and visualization tools for low-bit long-context testing on Apple Silicon.

Apple Silicon MLX port of TurboQuant — near-optimal KV cache quantization for local LLMs. Original: vivekvar-dl/turboquant (MIT), paper: arXiv:2504.19874

MLX implementation of Google's TurboQuant KV cache compression

3-bit KV cache compression for MLX on Apple Silicon (TurboQuant + PolarQuant)

No description available

No description available

Extreme weight + KV cache compression for LLMs on Apple Silicon (MLX implementation of Google's TurboQuant)

No description available