Implemented a script that automatically adjusts Qwen3's inference and non-inference capabilities, based on an OpenAI-like API. The inference framework can be sglang, or it can be adapted/modified to use vLLM

Platform-agnostic toolkit to spin up vLLM endpoints and submit high-throughput jobs (DataFrame or scripts) across Slurm and DGX Cloud Lepton.

Automated bash script to set up a high-performance environment on Ubuntu Linux with RTX5090, including installations of PyTorch, Unsloth, vLLM, Triton, Xformers. This script handles system dependencies, creates a Python virtual environment, compiles libraries from source, and verifies installations to ensure an optimal AI and deep learning setup.

CI scripts designed to build a Pascal-compatible version of vLLM.

DGX Spark setup and vLLM deployment scripts for Qwen, GPT-OSS, and Nemotron 3.

A terraform based bootstrap script which allows the standup of vllm + ray for Apple Silicon based workloads

Scripts to run large language models with text generation using vLLM.

My dev scripts and documents about vLLM development

Benchmarking scripts to run on RunPod to compare/contrast vLLM vs SGLang on the same prompts.

A script for transforming pixtral von HF transformers version to mistral/vLLM version

Ubuntu Server edition: automated setup script for Intel Arc Pro B70 GPU LLM inference server with vLLM tensor parallelism. 140 tok/s on 2x B70, 540 tok/s on 4x B70. For Windows, see arc-pro-b70-inference-setup-windows.

OpenAI Compatible API scripts for RunPod vLLM Worker

vLLM inference scripts with SLURM Apptainer

vLLM configuration script. Easily find an optimal configuration for your vLLM and GPU, evaluate VRAM usage, and token throughput.

🚀 Automated installation script for vLLM on HPC systems with ROCm support, optimized for AMD MI300X GPUs.

Containerised NVIDIA Dynamo with vLLM Backend Ready-to-use Docker environment for running NVIDIA’s Dynamo inference framework with vLLM. Includes pre-installed dependencies, service setup (etcd, nats-server), and example scripts for running prompts via batch commands. Streamlines LLM inference without local setup overhead.

A simple, configurable Bash script to benchmark and compare inference performance between Llama.cpp and vLLM using the OpenAI-compatible `/v1/chat/completions` API

Scripts for vllm-model-bash efforts

Scripts for serving vllm on multi node

Scripts for programatic launching of vllm backends at TACC

simple test scripts for RL between vllm and torchtitan

SERA (AI2 Open Coding Agent) setup scripts - vLLM deployment for GPU servers

Standalone LLM inference benchmarking pipelines on AMD GPUs using ROCm, vLLM, MAD, and data visualization scripts.

This repo contains a fine-tuned LLaMA 3.2B model served using vLLM and Docker. The project includes a custom OpenAI-style API endpoint, benchmarking scripts, performance metrics, and monitoring setup. Designed for low-latency inference and production-ready LLM deployment.

vllm sand vllm-ascend scripts

Small repo for storing vllm server related scripts and files.

No description available

No description available

No description available

No description available