local-llm-integration

Run local LLMs like Gemma, Qwen, and LLaMA on Android for offline, private, real-time chat and question answering with LiteRT and ONNX Runtime.

Run a <400ms latency Voice Agent on just 4GB VRAM. Fully offline, no API keys required. Optimized for GTX 1650 and edge robotics with zero-copy inference. (Apache 2.0)

🚀 A powerful Flutter-based AI chat application that lets you run LLMs directly on your mobile device or connect to local model servers. Features offline model execution, Ollama/LLMStudio integration, and a beautiful modern UI. Privacy-focused, cross-platform, and fully open source.

Local LLM chat panel for ComfyUI — LM Studio & Ollama, multi-session, vision support, Guide Materials

Distributed peer-to-peer LLM inference. Your prompt never leaves your device in clear text.

A premium, privacy-first, offline AI chat application for Android/iOS, integrating with Ollama via Termux. Experience powerful AI models directly on your device with a beautiful, customizable interface.

🖼️ Python Image and 🎥 Video Generator using LLM providers and models — built with Claude Code 💻 CLI

Local LLM proxy, DevOps friendly

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

An advanced, fully local, and GPU-accelerated RAG pipeline. Features a sophisticated LLM-based preprocessing engine, state-of-the-art Parent Document Retriever with RAG Fusion, and a modular, Hydra-configurable architecture. Built with LangChain, Ollama, and ChromaDB for 100% private, high-performance document Q&A.

A framework for using local LLMs (Qwen2.5-coder 7B) that are fine-tuned using RL to generate, debug, and optimize code solutions through iterative refinement.

A fully customizable, super light-weight, cross-platform GenAI based Personal Assistant that can be run locally on your private hardware!

Nova Studio - Windows desktop workbench for local LLM inference (vLLM / SGLang / llama.cpp via WSL2), bridging WSL engines to any OpenAI-compatible client

🤖 An Intelligent Chatbot: Powered by the locally hosted Ollama 3.2 LLM 🧠 and ChromaDB 🗂️, this chatbot offers semantic search 🔍, session-aware responses 🗨️, and an interactive Streamlit interface 🎨 for seamless user interaction. 🚀

An AI-powered assistant to streamline knowledge management, member discovery, and content generation across Telegram and Twitter, while ensuring privacy with local LLM deployment.

An autonomous AI agent for intelligently updating, maintaining, and curating a LightRAG knowledge base.

**Ask CLI** is a command-line tool for interacting with a local LLM (Large Language Model) server. It allows you to send queries and receive concise command-line responses.

UNOFFICIAL Simple LM Studio Web UI (Docker)

An AI Chat/Agent C# library with heavy focus on robust toolings, and swapping between LLM providers OpenAi, Gemini or a local Ollama.

Turns messy leads from any source (Facebook, Instagram, Google Forms, landing pages) into clean, deduplicated, scored leads in one schema. Each lead gets a 0–100 score, a plain-English reason, and a next action. Fixes its own code when it breaks. 100% free and runs fully local.