SoftPicksLocal LLM Tools
Explore the highest-rated open-source tools for running Large Language Models locally. Discover inference engines, GUI clients, and API wrappers for GGUF and safetensors models. Sorted by GitHub authority and active contributions.
Your local-first AI operator. Beats Hermes on GAIA L1 (69.8% vs 58.5%).
Open Dungeon — the first easy-to-use, fully local AI roleplay app. Story and inline scene images
Local-first healthcare AI: clinical NER & HIPAA PII de-identification that runs 100% on-device.
Persistent memory + local AI for coding agents. 1.7B–32B open-weight LLM fleet, cross-session Mind
Auto-tuned launcher for GGUF models on llama.cpp / ik_llama.cpp — OpenAI-compatible server with
Free macOS-native Cluely alternative. AI chat invisible to screen sharing. Ollama, OpenAI, Claude.
Ask questions about your Obsidian notes using local AI. Privacy-first RAG with Ollama, LM Studio, or
MCP server that saves Claude Code tokens by delegating bounded tasks to local or cloud LLMs. Works
AI-powered CLI tool: Transform trading research papers into QuantConnect algorithms
Curated real-world use cases for Hermes Agent — the self-improving AI agent from Nous Research.
Extend the Ollama API with dynamic AI tool integration from multiple MCP (Model Context Protocol)
InferrLM - On-device AI for iOS & Android
Fulloch - The Fully Local Home Voice Assistant
Portable multi-agent AI developer setup for Claude Code + Ollama. Role-based local LLM orchestration
A private Claude-Code-style coding agent for Apple Silicon — run chat, code, and local model
Turn your Android phone into an OpenAI-compatible LLM inference server — Fully local, private and
Hyprdots, Hyprland Omarchy, Quickshell, Agentic Local-Ai, Custom Integrated, Minimal
A simple experiment on letting two local LLM have a conversation about anything!
Enterprise-ready self-hosted AI assistant runtime with sandboxed execution, secure credentials,
Modern desktop application (Rust + Tauri v2 + Svelte 5 + Candle (HF)) for communicating with AI
⚡ A local, privacy-focused AI desktop assistant for Windows. Control your PC remotely via Telegram
More than just Karpathy’s LLM Wiki, 100% local with Ollama. Drop Markdown notes → AI extracts
Memory that learns what works.
Synthetic Autonomic Mind - An AI assistant for everyone.