vLLM
Install vLLM, the free Apache-2.0 engine for fast LLM inference and serving with an OpenAI-compatible API. Linux install commands and GPU needs.
Open source
- Linux
Tag
llm serving
1 program
Free games, apps and software, straight from official sources
Tag
1 program
Install vLLM, the free Apache-2.0 engine for fast LLM inference and serving with an OpenAI-compatible API. Linux install commands and GPU needs.
Open source