Back to search
Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offering scalability, extensibility, and high performance. It helps users quickly deploy models and provide services through HTTP/RPC interfaces.
Stars
169
Forks
11
Watchers
169
Open Issues
0
Overall repository health assessment
No package.json found
This might not be a Node.js project
No contributors data available
ADD: Add --no_loging param. Add cityhash dep for grps-trtllm.
0a31f16View on GitHub