GitHub Store
ddalcu

mlx-serve

ddalcu

A native LLM inference server optimized for Apple Silicon, supporting OpenAI and Anthropic-compatible APIs with no Python required, supporting both MLX and GGUF models, and including a macOS menu-bar app.

v26.11.2
Published at
7d ago
2,390
Stars
29,153
Download
Synced at
7d ago

Latest release

v26.11.2Nov 6, 2026, 02:15 AM
Downloads slow? Switch the source anytime above

README

Project Overview

mlx-serve is a native LLM inference server for Apple Silicon that exposes OpenAI- and Anthropic-compatible HTTP APIs. It runs any LLM — MLX-format models and every GGUF on HuggingFace (Qwen, Llama, Mistral, Gemma, DeepSeek, and thousands more) — and supports image, video, music, speech (with voice cloning), and 3D model generation.

Key Features

  • OpenAI- and Anthropic-compatible HTTP APIs
  • Supports both MLX and GGUF models
  • Supports image, video, music, speech (with voice cloning), and 3D model generation
  • MLX Core macOS menu-bar app with chat, agent mode, MCP tool calling, and model management
  • Ollama API compatible (/api/chat, /api/generate, etc.)

Target Audience

  • Apple Silicon Mac users
  • Developers who need local LLM inference
  • Users looking for an alternative to LM Studio

Quick Start

  1. Recommended: Download MLX Core.app (signed and notarized macOS menu-bar app)
  2. Homebrew install: brew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-serve && brew install --cask mlx-core
  3. CLI usage: mlx-serve run gemma4
  4. Server address: http://localhost:11234

Core Advantages

  • Optimized for Apple Silicon — faster than LM Studio
  • No Python, no cloud, no Electron
  • Supports OpenAI, Anthropic, and Ollama API compatibility
  • Open-source and free

Release history

Sort