llama.cpp-launcher

Project Overview

llama.cpp-launcher is a lightweight llama.cpp launcher built with Rust, providing a graphical interface to manage and run llama.cpp servers. It supports visual configuration, RPC mode, preset management, real-time logs, and a multi-language interface.

Key Features

• Cross-platform GUI: Built on eframe/egui
• Visual configuration: Configure llama.cpp server parameters (port, threads, GPU layers, etc.)
• File browser: Select model files (.gguf format)
• Preset system: Save and load common configurations
• Real-time log display: File logging support
• Chinese/English UI: Auto-detects system language
• Auto-start on boot (Windows)
• GPU layer management: Auto/All/Manual modes
• Auto-calculates max context from available VRAM
• RPC mode: Distributed inference
• Web UI integration: Supports opening the web client
• Model file color label parsing: Auto-detects parameter count, quantization type, etc.
• Launch command preview: Read-only display of the final command

Target Audience

• Users who need to run llama.cpp servers on Windows or Linux
• Users who prefer a GUI over the command line for managing llama.cpp
• Developers and tech enthusiasts interested in local LLM inference

Quick Start

1. Download the llama.cpp release and extract it
2. Place the launcher in the same directory as llama.cpp
3. Launch the application and select the Server panel
4. Click "Auto Detect" and the program will automatically find the llama.cpp server executable
5. Configure server parameters and select a model file (.gguf format)
6. Click "Start Server"; once the "Open Web Client" button lights up, click it to open the Web interface

Core Advantages

• Graphical interface, no need to memorize complex command-line parameters
• Supports RPC mode (distributed inference)
• Preset system and model file color label parsing for convenient management
• Chinese/English UI with automatic system language detection

System Requirements

• OS: Windows, Linux
• Additional dependencies: Rust 1.70+ for building from source; Visual Studio Build Tools or MinGW on Windows; X11 development libraries on Linux