Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

DevoxxGenie is a plugin for IntelliJ IDEA that uses local LLM's (Ollama, LMStudio, GPT4All, Jan and Llama.cpp) and Cloud based LLMs to help review, test, explain your project code. Latest version now also supports Spec Driven Development with CLI Runners.

A basic startup guide on running LLMs on android locally or using an external ollama server

prr - command-line LLM prompt runner

Zero-dependency browser automation CLI. 70+ commands, 10 test assertions, smart commands (click/fill by text — no LLM needed). MCP server for AI agents with 500x fewer tokens. Extract, observe, script runner. 50KB, pure CDP.

Actor based multi-llm registry + runner.

The code for HerO: a fact-checking pipeline based on open LLMs (the runner-up in AVeriTeC)

OpenAI-sublime-text package core package implemented in rust

Tools like web browser, computer access and code runner for LLMs

Step by step guide to setup the ai assistant in marimo notebooks using local LLM models running with Docker Model Runner

A flexible, extensible AI agent backend built with NestJS—designed for running local, open-source LLMs (Llama, Gemma, Qwen, DeepSeek, etc.) via Docker Model Runner. Real-time streaming, Redis messaging, web search, and Postgres memory out of the box. No cloud APIs required!

This provides sample codes that uses Microsoft.Extensions.AI for locally running LLMs through Docker Model Runner, Foundry Local, Hugging Face and Ollama

An open-source runner for LLM workflows that makes prompts testable, deployments repeatable, and costs observable

👐🔌 LLM Tool Runner - Chat with your APIs - Abstraction layer over Function Calling - https://openplugin.com/

Saraf AI is a fully local assistant built with Next.js and Docker. It connects seamlessly to LLMs via Docker Model Runner using Docker Compose.

Materials for the ODSC-AI 2025 workshop - Running LLMs Locally with Docker Model Runner

Experimental AI-Based UI Test Runner for Xcode using LLMs

Use LLMs to do semantic testing of website content and functionality using Jest, Testcontainers and Docker Model Runner

Open-source, encrypted, Tor-powered AI model runner. Privacy-first alternative to LM Studio. Run LLMs locally with E2E encryption, TLS API server, Tor routing. Zero telemetry. GPL-3.0.

CMake-based large language model library with Arm® KleidiAI™ backend support

The code for HerO 2: an efficient fact-checking pipeline based on open LLMs (the runner-up in AVeriTeC 2.0)

aida is the oneshot runner for LLM-generated shell commands

an ooba booga (or any llm runner and hoster thing) for discord.py

MCP server for orchestrating LLM agents as task runners with multi-transport support, pluggable storage, and production monitoring

DOT-graph AI pipeline runner with LLM tasks, human gates, conditional branching, parallel execution, real-time web dashboard, and REST API.

Library alternatif untuk mengutilisasi LLM ke programming NodeJS/Javascript

A CLI thinking partner that critiques and refines its own responses. Local LLMs only.

A Streamlit-based web interface to run and compare multiple LLMs in parallel using Ollama, with support for dynamic model selection, prompt input, and side-by-side response comparison.

No description available

Web application for llm test runner.