vLLM Download: A High-Throughput Engine for Serving LLMs
vLLM · for Linux
Developer: vLLM project (opens in a new tab)
- Linux
The download link points to the program’s official source; the file is not hosted on this site. Link address: docs.vllm.ai
Link not working? Let us know
vLLM is a free, open-source engine for fast inference and serving of large language models, with an OpenAI-compatible API server. It runs on Linux with Python 3.11–3.14, supports NVIDIA, AMD and Intel GPUs as well as CPUs, and is licensed under Apache 2.0.
When you need to serve a language model to many users at once, raw speed matters. vLLM was originally developed at UC Berkeley’s Sky Computing Lab and is widely used to run open models on GPU servers. It has about 96,000 stars on GitHub as of October 2026.
What does vLLM do?
- Fast serving: High-throughput inference with PagedAttention and continuous batching of incoming requests.
- OpenAI-compatible server: Start a model with one command and use it from any OpenAI-compatible client; an Anthropic Messages API is also available.
- Broad hardware: NVIDIA, AMD and Intel GPUs, plus x86, ARM and PowerPC CPUs; Apple Silicon through the vLLM-Metal plugin.
- Hugging Face models: Serve more than 200 model architectures directly by their Hugging Face name.
- Quantization: FP8, INT8, INT4, GPTQ/AWQ, GGUF and more.
Who is it for?
vLLM is aimed at developers and companies running models on Linux GPU servers. For simple desktop chatting, tools such as Jan are easier.
Quick start on Linux
The official quick start uses uv with an NVIDIA GPU:
uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm --torch-backend=auto
vllm download-kernels
vllm download-kernels installs precompiled kernels so startup is faster; run it again after every install or upgrade. Then start a server:
vllm serve Qwen/Qwen2.5-1.5B-Instruct
The server listens at http://localhost:8000 (change it with --host and --port); check it with:
curl http://localhost:8000/v1/models
System requirements
- Operating system: Linux. Windows is not supported natively; the docs suggest WSL.
- Python: 3.11 to 3.14.
- NVIDIA: Compute capability 7.5 or higher, such as T4, RTX 20 series, A100, L4 or H100.
- AMD: ROCm 6.3 or later; prebuilt wheels need Python 3.12.
- Intel: Data Center GPUs and Arc GPUs, with Python 3.12.
vLLM is free and open source under the Apache 2.0 license.
Pros
- Very fast serving for many users at once
- OpenAI-compatible API out of the box
- Supports NVIDIA, AMD and Intel GPUs
Cons
- Linux only; Windows needs WSL
- Aimed at servers rather than casual desktop use
Frequently Asked Questions
Does vLLM run on Windows?
Not natively. vLLM targets Linux; on Windows the documentation suggests using WSL.
Which NVIDIA cards does vLLM support?
Cards with compute capability 7.5 or higher, for example the T4, RTX 20 series and newer, A100, L4 and H100.
vLLMOpen source
Download (opens in a new tab)
