cross-platform-llm-client

Project Overview

PrivateLM is a production-ready, cross-platform AI chat client built with Flutter. It unifies local on-device LLM inference (Android) with cloud API access, giving users full control over how their models run.

Key Features

• Local inference on Android: Download and run GGUF models directly on your phone using GPU-accelerated inference (Vulkan). No internet required after download
• Cloud API fallback: Seamlessly switch to OpenAI, Anthropic, Google Gemini, or Kimi
• Multimodal chat: Send text and images; vision support works with both local models and cloud providers
• Persistent sessions: All chats and settings stored locally via Hive
• Smart auto-configuration: On first launch, the app detects your device's RAM and recommends optimal settings
• Task management: Dedicated task view for structured AI-assisted workflows

Target Audience

• Privacy-conscious users who want to run LLMs locally
• Developers who need both cloud and local model access
• Users who want uncensored models

Quick Start

1. Download from GitHub Releases for your platform
2. On first launch, the app auto-detects device capabilities and recommends settings
3. Download GGUF models for local inference or configure cloud APIs
4. Start chatting

Core Advantages

• Seamless switching between local inference and cloud APIs
• GPU-accelerated local inference (Vulkan)
• All data stored locally for privacy
• Built with Flutter for cross-platform support

System Requirements

• OS: Android (local inference), iOS, macOS, Windows, Linux
• Additional Dependencies: Vulkan (Android local inference)