Docker Model Runner now supports vllm-metal, a new backend enabling vLLM inference on macOS with Apple Silicon's Metal GPU. Developed collaboratively by Docker and the vLLM project, vllm-metal unifies MLX and PyTorch under a single compute pathway, leveraging Apple Silicon's unified memory for zero-copy tensor operations and paged attention for efficient KV cache management. The backend runs natively on the host (not in containers) due to Metal GPU access requirements, and is compatible with MLX-format safetensors models from the mlx-community on Hugging Face. Docker has open-sourced vllm-metal under the vLLM GitHub organization. Benchmarks show llama.cpp is ~1.2x faster in raw throughput, but vllm-metal brings the full vLLM ecosystem (OpenAI-compatible API, scheduler, batching) to Mac. Docker Desktop 4.62+ is required, and installation is a single command.
Table of contents
What is vllm-metal?How vllm-metal worksWhich models work with vllm-metal?vLLM everywhere with Docker Model RunnerGet startedGiving Back: vllm-metal is Now Open SourceHow does vllm-metal compare to llama.cpp?Share this post