A lightweight LLaMA.cpp HTTP server Docker image based on Alpine Linux.

Latest llama.cpp in a containerized server + langchain support

Docker images for easier running of llama-cpp-python server

Docker image for Lllama2 server which works out of the box on the GPU

llama.cpp server + small language model in Docker container

This project is build off of the lit-llama project found here: https://github.com/Lightning-AI/lit-llama The main functionality improvment is encapsulating the LLAMA into a docker container with a client / server message queue for interaction.

llama-server in Docker.

Docker setup for llama.cpp server with router mode, supporting multiple GGUF models with lazy loading

Project starter for building Airflow that integrates with local llama.cpp server

Docker template for connecting Airflow with dockerized llama.cpp servers

Serving open source models of your choice in as a docker container using llama-cpp-python's OpenAI compatible server

DeployIQ is a application that simplifies deploying open-source LLMs (like Llama 3, Mistral, and Falcon) on cloud platforms (AWS, GCP, Azure), bare-metal servers, or local Docker environments, with automated GPU provisioning, scaling, and secure API exposure, while integrating monitoring and logging for seamless inference management.

Egyptian AI Assistant is a lightweight local chatbot that runs fully offline using llama-server and a GGUF model. It includes a modern Next.js chat UI, a FastAPI backend, and Docker support. The backend enforces Egyptian Arabic responses, The backend enforces Egyptian Arabic responses with optional voice endpoints that can be enabled when needed.

No description available

No description available

No description available

No description available

llama.cpp CUDA Server Docker Image

Docker Image for Llama.cpp Server

CUDA-accelerated llama.cpp Docker inference server

A dockerized option to benchmark your Llama.cpp server.

No description available

Simple FastAPI server wrapped in Nvidia docker for Llama inference

Docker Server for run meta-llama/Llama-3.3-70B-Instruct with a HUGGINGFACE_API_KEY

Docker Server for run meta-llama/Llama-3.1-8B-Instruct with a HUGGINGFACE_API_KEY

Find, download model from HF and change model in llama.cpp server (Docker)

Built from upstream llama.cpp as a CUDA-enabled llama-server container for local LLM inference with a simple, reproducible Docker workflow.

A Dev Container + Docker Compose setup to easily run a crewai using llama.cpp server

Run llama.cpp server in Docker with GPU support. Model-independent, swap models by changing one line.

A project to deploy a llama.cpp server with a web UI to AWS using Docker and Terraform.