llama.cpp Download: The Open-Source Engine That Runs Language Models on Any Hardware
llama.cpp 0.7.0 · for Windows, macOS and Linux
Developer: ggml (opens in a new tab)
- Windows
- macOS
- Linux
The download link points to the program’s official source; the file is not hosted on this site. Link address: github.com
Link not working? Let us know
llama.cpp is an open-source inference engine that runs large language models locally in C/C++ with no extra dependencies. The latest stable version is 0.7.0, released on October 15, 2026; it is developed by the ggml team, MIT-licensed, and runs on Windows, macOS and Linux.
In the world of local AI, the same engine runs behind the scenes of many tools: llama.cpp. The project is a lightweight C/C++ library plus a set of command-line tools, written so that language models can run on ordinary computers, even on the CPU alone. Its GitHub repository has roughly 133,000 stars as of October 2026.
What does llama.cpp do?
llama.cpp downloads and runs models in the GGUF format and gives you two main ways to use them: chat directly in the terminal, or serve the model as an OpenAI-compatible API. Server mode also includes a built-in web interface, so you can chat from your browser and connect apps that support the OpenAI API to your local model.
Key features
- No dependencies: Written in plain C/C++; it needs no Python environment or heavy libraries.
- Broad hardware support: Accelerates on NVIDIA (CUDA), AMD (HIP/ROCm), Intel (SYCL), Vulkan-capable cards and Apple Silicon (Metal), and makes use of AVX, AVX2 and AVX512 instructions on x86 CPUs.
- Quantization: Compressed model versions from 1.5-bit to 8-bit greatly reduce memory needs.
- Hybrid execution: If a model doesn’t fit in video memory, it can run part of it on the graphics card and the rest on the CPU.
- Hugging Face integration: Downloads models from Hugging Face by name and runs them directly.
Who is it for?
It suits developers who want to control local models down to the finest setting, people who want to run models even on weak hardware, and programmers who want to embed local AI in their own apps. If you’re not comfortable with the command line, Llama (llama.app), an app the same team builds together with Hugging Face, is an easier option that runs from the menu bar on Windows 11 and macOS.
How to install
The official install scripts:
# macOS and Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows (PowerShell)
irm https://llama.app/install.ps1 | iex
If you use a package manager:
# Windows
winget install llama.cpp
# macOS and Linux
brew install llama.cpp
Alternatively, download a prebuilt package for your operating system and graphics card (for example the CPU, CUDA or Vulkan build for Windows) from the GitHub releases page that the Download button opens. Besides stable releases, new builds are published almost every day.
First steps
To download a small model from Hugging Face and start chatting in the terminal:
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
To serve the same model as an OpenAI-compatible server:
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
By default the server listens on http://127.0.0.1:9931; open that address in your browser to use the built-in web interface.
Hardware requirements
Because llama.cpp also runs on the CPU alone, there is no hard minimum. The memory you need depends on the size and quantization level of the model you choose: small models run comfortably on an ordinary laptop, while large models need tens of GB of memory. A supported graphics card or an Apple Silicon chip speeds things up considerably.
License
llama.cpp is MIT-licensed; it is free and can be used in commercial projects too. Keep in mind that each model you run has its own license.
Pros
- Very light, dependency-free and fast
- Extremely broad hardware support, from CPUs to GPUs
- OpenAI-compatible server with a built-in web interface
Cons
- Settings are too technical for beginners
- Runs only GGUF-format models
Frequently Asked Questions
What is the difference between llama.cpp and Ollama?
llama.cpp is the low-level engine that runs models and puts every setting in your hands. Ollama is a more ready-to-use tool that makes downloading and managing models easier. Those who want fine control pick llama.cpp; those who want a quick start pick Ollama.
Which model files does llama.cpp run?
llama.cpp runs models in the GGUF format. Many open models have ready-made GGUF versions on Hugging Face that can be downloaded directly with the -hf option.
llama.cppOpen source
Download (opens in a new tab)
