LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Streamer-Sales 销冠 —— 卖货主播 LLM 大模型🛒🎁，一个能够根据给定的商品特点从激发用户购买意愿角度出发进行商品解说的卖货主播大模型。🚀⭐内含详细的数据生成流程❗ 📦另外还集成了 LMDeploy 加速推理🚀、RAG检索增强生成 📚、TTS文字转语音🔊、数字人生成 🦸、 Agent 使用网络查询实时信息🌐、ASR 语音转文字🎙️、Vue 生态搭建前端🍍、FastAPI 搭建后端🗝️、Docker-compose 打包部署🐋

Llama3-Tutorial（XTuner、LMDeploy、OpenCompass）

Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)

gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR、TTS、文生图、图片编辑和文生视频的开源框架。

基于《西游记》原文、白话文、ChatGPT生成数据制作的，以InternLM2微调的角色扮演多LLM聊天室。 本项目将介绍关于角色扮演类 LLM 的一切，从数据获取、数据处理，到使用 XTuner 微调并部署至 OpenXLab，再到使用 LMDeploy 部署，以 openai api 的方式接入简单的聊天室，并可以观看不同角色的 LLM 互相交流、互怼。

Deploying LLMs offline on the NVIDIA Jetson platform marks the dawn of a new era in embodied intelligence, where devices can function independently without continuous internet access.

In this fast-paced world, we all need a little something to spice up life. Whether you need a glass of sweet talk to lift your spirits or a dose of sharp retorts to let off steam, FunGPT has got you covered 🎉!

DeepSparkInference has selected 216 inference models of both small and large sizes. The small models cover fields such as computer vision, natural language processing, and speech recognition; the LLMs involve various frameworks including vLLM, TGI and LMDeploy. This repository is the mirror of Gitee.

Self-host LLMs with LMDeploy and BentoML

This project is specifically developed for V100, based on lmdeploy 0.12.1, and supports mainstream open-source models from Q4 2025 to Q1 2026. It does not account for compatibility with other architectures and has only been tested on an 8-card V100 32GB setup.

Nightly Build for LMDeploy

Bench360 is a modular benchmarking suite for local LLM deployments. It offers a full-stack, extensible pipeline to evaluate the latency, throughput, quality, and cost of LLM inference on consumer and enterprise GPUs. Bench360 supports flexible backends, tasks and scenarios, enabling fair and reproducible comparisons for researchers & practitioners.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

No description available

根据lmdeploy官方pr 4389分支，在其对turbomind基础上，修改部分文件适配2080ti、T10等sm75架构，仅支持文本

使用lmdeploy

run Llama3.1 with Lmdeploy

Examples of serving LLM on Modal.

A multi-modal Large Language Model (LLM) application deployed on Modal.com that can process images and generate text descriptions using InternVL with LMDeploy.

Predictive GPU failure detection for inference startups. S++9 Blackwell thermal: 12-min advance warning before throttling. 125M events/sec throughput. Framework-agnostic (vLLM, SGLang, LMDeploy).

lmdeploy-dev

No description available

No description available

No description available

No description available

No description available

第五课作业

No description available

No description available