llmfit

Project Overview

llmfit is a command-line tool that inspects your CPU, system RAM, GPU, VRAM, and accelerator configuration to recommend open-source LLMs that will run well on your hardware. It supports interactive TUI, web dashboard, and REST API.

Key Features

• Hardware auto-detection: Detects CPU cores, system RAM, GPUs, VRAM, and unified memory architecture
• Model compatibility engine: Analyzes model parameter counts, context lengths, and quantization formats (GGUF, AWQ, GPTQ, EXL2)
• Interactive TUI & Web dashboard
• REST API endpoint: For integration into orchestrators and automated pipelines
• Multi-platform: macOS (Apple Silicon & Intel), Linux (x86_64 & ARM64), Windows (x86_64)
• Benchmark & share: Measure real tok/s on your hardware and contribute back

Target Audience

• Developers and researchers who want to run LLMs locally
• Engineers who need to select the best model for specific hardware
• Teams self-hosting LLM infrastructure

Quick Start

1. Install llmfit
2. Run the tool to auto-detect your hardware
3. View recommended models and performance estimates
4. Optionally run benchmarks for real performance data

Core Advantages

• Hundreds of models & providers — one command to find what runs on your hardware
• Supports multi-GPU, MoE architectures, and multiple quantization formats
• Three interfaces: TUI, web dashboard, and REST API
• Real benchmark data can be contributed back to the community

System Requirements

• OS: macOS (Apple Silicon & Intel), Linux (x86_64 & ARM64), Windows (x86_64)