Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

🚀 Next Generation Multi-tenant AI One-Stop Solution. Builtin Admin & Billing System. Enterprise-Grade Unified LLM Gateway Support for 200+ Models And 35+ Providers, Load Balacing w/ Priority-base Routing, Cost Management, Chat Share, Cloud Sync, Credit/Subscription Billing, All File Parsing, Web Search, Built-in Model Cache.

Plano is an AI-native proxy and data plane for agentic apps — with built-in orchestration, safety, observability, and smart LLM routing so you stay focused on your agents core logic.

The agent-native LLM router for OpenClaw. 41+ models, <1ms routing, USDC payments on Base & Solana via x402.

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality

Smart LLM Routing for OpenClaw. Cut Costs up to 70% 🦞🦚

14-stage Fusion Pipeline for LLM token compression — reversible compression, AST-aware code analysis, intelligent content routing. Zero LLM inference cost. MIT licensed.

OmniRoute is an AI gateway for multi-provider LLMs: an OpenAI-compatible endpoint with smart routing, load balancing, retries, and fallbacks. Add policies, rate limits, caching, and observability for reliable, cost-aware inference.

LLMRouter: An Open-Source Library for LLM Routing

Permanent Free LLM API List (API Keys) 😎🔑

Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.

WilmerAI is one of the oldest LLM semantic routers. It uses multi-layer prompt routing and complex workflows to allow you to not only create practical chatbots, but to extend any kind of application that connects to an LLM via REST API. Wilmer sits between your app and your many LLM APIs, so that you can manipulate prompts as needed.

End-to-end AI-powered academic paper writing system — from idea generation and literature search to experiment execution, result backfill, and LaTeX paper compilation. Supports multi-LLM routing, SSH remote training, incremental sync, and anti-AI-detection writing style.

The Public ESPN API documentation provides a complete list of undocumented API endpoints, URL parameters, and JSON response formats across 20+ sports (NFL, NBA, MLB, NHL). Includes live curl examples, routing guides, and a Django REST API service for live scores, standings, and news. Optimized for search, developers, AI, and LLMs.

Open-source LLM router & AI cost optimizer. Routes simple prompts to cheap/local models, complex ones to premium — automatically. Drop-in OpenAI-compatible proxy for Claude Code, Codex, Cursor, OpenClaw. Saves 40-70% on AI API costs. Self-hosted, no middleman.

Access models from OpenAI, Groq, local Ollama, and others by setting llm-router as Cursor's Base URL

Lightweight LLM firewall: masks PII, routes calls, leaves zero trace.

Multi-agent orchestration middleware that coordinates between users, AI agents (LLMs), and business systems. It provides intelligent routing, conversation management, and API orchestration for building sophisticated AI-powered applications.

Tutorial for building LLM router

Route LLM requests to the best model for the task at hand.

[DAI 2025] Beyond GPT-5: Making LLMs Cheaper and Better via Performance–Efficiency Optimized Routing

A local LLM router that intelligently dispatches AI requests to the right model — saving cost without sacrificing quality.

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

🚀 AIRouter - 智能AI路由器：为多个LLM提供商提供统一API接口，支持负载均衡、故障转移和智能路由 | Intelligent AI Router with unified API interface, load balancing, and smart routing for multiple LLM providers

The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across SGLang, vLLM, TRT-LLM, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.

[NeurIPS'25] Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning

A curated list of awesome works in Routing LLMs paradigm (👉 Welcome to submit your contributions to this code repository)